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The following articles represent Test Center holdings to date in the area of assessment alternatives 
in mathematics. Presence on the list does not necessarily imply endorsement. Articles are 
included to stimulate thinking and provide ideas. Some of the entries are formal assessments, and 
are intended mainly for the classroom. For more information, contact Matthew Whitaker, Test 
Center Clerk, at (503) 275-9582, Northwest Regional Educational Laboratory, 101 SW Main, 
Suite 500, Portland, Oregon 97204, e-mail: testcenter@nwrel.org. To purchase a copy of this 
bibliography, please call NWREL's Document Reproduction Service at (503) 275-9519. 

Alberta Education. Diploma Examinations Program-Mathematics 30, 1991. Available 
from: Learning Resources Distributing Centre, 12360 - 142 St., Edmonton, 
AB T5L 4X9, Canada, (403) 427-2767, fax (403) 422-9750. 

Alberta Education develops diploma examinations in several course areas. These, combined 
with school-awarded "marks" are used to assign credit for the courses. The Mathematics 30 
examination has three parts: multiple-choice, "numerical response" (students "bubble" their 
answers onto the scan sheet), and written response. The test appears to only assess 
knowledge of the subject area rather than problem solving, communication, reasoning, etc. 
Examinations are given locally under controlled conditions. Papers are scored centrally. 
Scoring appears to be based on the correctness of the answer. 

The mathematics test covers advanced algebra, geometry, trigonometry, and number theory. 



(TC# 500.3DIPEXP) 



Alberta Education. Diagnostic Mathematics Program, 1990. Available from: The Learning 
Resources Distributing Centre, 12360 - 142 St., Edmonton, AB T5L 4X9, Canada, 
(403) 427-2767. 

The Diagnostic Mathematics Program provides assessment and instructional ideas and 
materials for five goals areas (numeration, operations/properties, measurement, geometry, and 
problem solving) at six grade levels (1-6). There are ten handbooks designed for classroom 
teachers: five goals areas times two levels (grades 1-3 are bound together in the same books, 
as are grades 4-6). Each handbook contains "observation checklists," "structured interviews," 
and written tests for several subskills within each skill area. For example, the numeration 
handbook in grade 3 has separate sections for: place value/whole numbers, decimals, 
fractions, and ordering whole numbers. 

"Observations" require teachers to make a judgment of the skill level (strong, adequate, or 
weak) of the student in tach area to identify those students for whom further evaluation is 
necessary. Some assistance is given on the criteria for making these judgments. 

"Interviews" require students to solve problems or answer questions presented orally. These 
are the same types of problems as on the written tests. Except for a few problems in the 
problem-solving handbook, problems are atomistic-each problem is designed to assess a 
different skill. Except for a few problems in the problem-solving handbook, responses are 
short answer and are scored right/wrong. Total correct is used to assign a rating of "strong," 
"adequate," or "weak" for each skill. No rationale is provided for the cut scores given for 
each rating. In problem-solving, a few problem solutions are scored judgmentally on a scale 
of 0-2 in three areas: "understanding the problem," "developing and carrying out the plan," 
and "looking back." A generalized scoring guide is adapted to each specific problem. 

No technical information is provided. 

(TC# 500.3DIAMAP) 

Algina, James, and Sue Legg (Eds.). Special Issue: The National Assessment of 
Educational Progress. Located in: Journal of Educational Measurement 29, 
Summer 1992. 

This special issue of JEM discusses the National Assessment of Educational Progress 
(NAEP)--history, specification of content and design of assessments for 1992 and beyond, 
how students are sampled, and how results are reported. Although some articles are 
somewhat technical, the general pieces on NAEP's history, and the design of current 
assessments will be interesting to the general readership. 

The current plans for math include: 

1 . Use of calculators for about 70 percent of the test 

2. Estimation skills tasks using an audiotape. 
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3. Yes/no questions to determine the extent to which students understand the same 
information when it is presented in different forms. 

4. Constructed response questions in which students are asked to documen . their solutions 
by drawing their answers, writing explanations, or providing their computations. 

Scoring guides for open-ended questions are task-specific. Some examples are provided. 

(TO 150.6JEM292) 

Appalachia Educational Laboratory. Alternative Assessments in Math and Science: Moving 
Toward a Moving Target, 1992. Available from: Appalachia Educational Laboratory, 
PO Box 1348, Charleston, WV 25325, (304) 347-0400. 

This document reports on a two-year study by the Virginia Education Association and the 
Appalachia Educational Laboratory. In the study, 1 1 pairs of K-12 science and math teachers 
designed and implemented new methods of evaluating student competence and application of 
knowledge. 

Teachers who participated in the study found that the changes in assessment methods led to 
changes in their teaching methods, improvements in student learning and better student 
attitudes. Instruction became more integrated across subjects and shifted from being etcher- 
driven to being student-driven. Teachers acted more as facilitators of learning rather than 
dispensers of information. 

Included in the report is a list of recommendations for implementing alternative assessments, a 
list of criteria for effective assessment, and 22 sample activities (with objectives, tasks, and 
scoring guidelines) for elementary, middle, and high school students, all designed and tested 
by the teachers in the study. 

Most activities have performance criteria that are holistic and task-specific. No technical 
information or sample student work is included. 

(TC# 600.3ALTASM) 



Aurora Public Schools. Performance Assessments in Science and Mathematics, 1992. 
Available from: Strategic Plan Facilitator, Aurora Public Schools, Department of 
Instructional Services, 15751 E. 1st Ave., Suite 220, Aurora, CO 80011, (303) 340-0861, 
fax: (303) 340-0865. 

The author has provided three examples of the types of assessments being developed by 
teachers in Aurora Public Schools: developing an analogy for the major anatomical and 
physiological components of a typical eukaryotic cell, recommending a decision concerning 
the future use of a medical technology in human biology, and collecting and analyzing a data 
set. These examples, for secondary students, include a description of the task, prerequisite 
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student experiences, and criteria forjudging student performance on the task. Students work 
in groups of two to four. The assessments are mostly for classroom use. 

Performances are evaluated along several dimensions including content, complex thinking, 
decision making, and collaborative working. Most of the rubrics are task specific and 
emphasize relative quality. For example, a "4" score for complex thinking on the medical 
technology task is: "The student clearly and completely identified the criteria by which the 
alternatives were assessed. The criteria were presented in detail and reflected an unusually 
thorough understanding and concern for the repercussions of the decision." The collaborative 
worker rubric is generic and more descriptive; a "4" is "The student expressed ideas clearly 
and effectively; listened actively to the ideas of others; made a consistent effort to ensure that 
ideas were clearly and commonly understood; accurately analyzed verbal and non-verbal 
communications; solicited and showed respect for the opinions of others." 

No technical information nor sample student responses are included. 

(TC# 000.3SCEMAP) 



Badger, Elizabeth. On Their Own: Student Response to Open-Ended Tests in Mathematics, 
1989-1991. Available from: Massachusetts Educational Assessment Program, 
Massachusetts Department of Education, 1385 Hancock St., Quincy, MA 02169, 
(617) 770-7334. 

The document we received contained assessment materials for grades 4, 8, and 1 2 from three 
years (1988-1990) in four subject areas (reading, social studies, science and math). This entry 
describes the math portions of the assessments. The 1988 and 1990 materials described open- 
ended test items in which students had to solve a problem and then explain their answer. In 
1988 eight problems were administered to each of the three grades (some problems were 
repeated between grades). In 1990, ten problems were administered. These problems 
emphasized the major areas of patterns/relationships, geometry/measurement, and 
numerical/statistical concepts. All problems were done individually in written format. 
Problems were distributed in such a way that different students responded to different 
questions. Responses were scored both for correctness of solution and for quality of the 
explanation. No specific criteria forjudging quality of explanation were given. Many 
examples of student responses illustrating various conclusions are included. 

In 1989, a sample of 2,000 students was assigned one of seven performance tasks (four in 
math required manipulatives) to do in diads. Each pair was individually watched by an 
evaluator. Each evaluator could observe between six and ten pairs each day. It took 65 
evaluators five days to observe the 2,000 performances. Evaluators were to both check off 
those things that students did correctly (e.g., measured temperature correctly), and record 
observations of students' conversations and strategies as completely as possible. A sample 
checklist of skills includes: measuring, proportional reasoning, equivalency, numeration, 
attitude, and planning/execution. 
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Some information on results for all the assessments is provided: percentages of students 
getting correct answers, using various strategies, using efficient methods, giving good 
explanations, etc., depending on the task. Many examples of student responses illustrating 
these various points are provided. No technical information about the assessments themselves 
is provided. 

(TC# 500.3ONTHOM) 

Bagley, Theresa, and Catarina Gallenberger. Assessing Students' Dispositions: Using 
Journals to Improve Students' Performance. Located in: The Mathematics Teacher 85, 
November 1992, pp. 660-663. 

In this article, the authors discuss the use of journals to elicit behavior that can be examined 
for high school students' attitude toward math, making mathematical connections, and 
understanding. They present many questions, tasks, and instructions for getting students to 
self-reflect, and provide good, practical suggestions for managing the process. However, the 
authors do not provide criteria for examining student responses (i.e., what to look for in 
responses that are indicators of attitude, connections or understanding), so the procedure is 
informal. The procedure will only be useful to the extent that users have the expertise to 
know what to look for in responses. 

(TC# 500.6ASSSTD) 

Barton, Paul E. National Standards for Education: Wiiat They Might Look Like; A 
Workbook, 1992. Available from: Educational Testing Service, Policy Information 
Center, Mail Stop 04-R, Princeton, NJ 08541, (609) 734-5694. 

This monograph presents examples of standards from eight different projects. The intent is to 
illustrate and document some existing standards, help policymakers sharpen their thinking 
about standards, and help people develop common concepts of standards. The eight samples 
come from NCTM Math Standards, Project 2061 in science. Advanced Placement US 
History, NAEP Science Objectives, Toronto Benchmarks in math and language arts, NAEP 
Geography Objectives, National Curriculum in England and Wales in math, and Florida 
Department of Education on general definitions of terms. 

(TC# 500.5NATSTE) 

Baxter, Gail P., Richard J. Shavelson, Sally J. Herman, Katharine A. Brown, and James R. 
Valadez. Mathematics Performance Assessment: Technical Quality and Di. e Student 
Impact Located in: Journal for Research in Mathematics Education 24, 1993, 
pp. 190-216. 

The authors developed 4 1 hands-on tasks to measure three categories of sixth-grade student 
competencies: measurement (seven tasks), place value (3 1 tasks), and probability (three 
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tasks). An example of a measurement task is "describe the object" in which students had to 
write a description of an object that someone else could use to draw the object. Sixteen of the 
place value tasks were "card shark" in which students were dealt cards with four numbers 
(e.g., 6000, 100, 60 and 2). They had to put the cards together to form a specified number, 
read the number aloud, and name the place value of a particular digit. An example of a 
probability task was "spin it" in which students were given a spinner with eight sections (four 
orange, three yellow, and one green). They had to predict which color the pointer would land 
on most or least often, predict the outcome of 32 spins, and carry out the experiment and 
graph the results. 

Responses were scored either by degree of "correctness" or, in the case of the communication 
items (e.g., describe an object), holistically for general quality of the response. The tasks and 
criteria were described only in general terms; further information would have to be obtained 
from the authors in order to actually reproduce the assessment. 

Tasks were pilot tested wit'i 40 sixth graders (Anglo and Hispanic) from two types of 
instructional settings: hands-on and traditional. Results showed: raters using this type of 
rating scheme can be trained to be very consistent in their scoring; the assessments are costly 
and time-consuming; a considerable number of tasks need to be administered to provide a 
reliable estimate of a student's level of achievement; student performances on the hands-on 
tasks differed by the type of instructional setting (evidence of validity); and there was 
differential scoring on the part of Hispanics, leading to some equity concerns. 

(TC# 500.6MATPEA) 

Brasweli, James. Overview of Changes in the SA T Mathematics Test in 1994. {SA T 

Mathematics—Student Produced Responses], 1991. Available from: Educational Testing 
Service, Rosedale Rd., Princeton, NJ 08541, (609) 734-5686. 

This was a paper presented at the annual meeting of the National Council on Measurement in 
Education, April 5, 1991, Chicago. 

Currently, the SAT-Math consists of two parts: regular multiple-choice and quantitative 
comparison (e.g., solution A is larger than, smaller than, or equal to, solution B, or cannot be 
determined). A third part called "student-produced responses" will be included on the PSAT 
in 1993 and the SAT in 1994. In this part, students will solve problems that have integer, 
fractional, or decimal solutions in the range 0 to 9999. A grid is provided for students to 
enter their actual answer. Some problems will have more than one right answer or can be any 
value in a range. For these problems, a correct response is recorded if the student answer is 
one of the accepted answers. Of the 55-60 items on the test, 10-15 will be in this format. 

The materials include a couple of examples of this type of item. 

(TC# 500.3SATMAS) 
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Burton, Grace, Terrence Coburn, John Del Grande, et al. Addenda Series, Grades K-6: 
Curriculum and Evaluation Standards for School Mathematics, 1991-92. Available from: 
National Council of Teachers of Mathematics, 1906 Association Dr., Reston, VA 22090, 
(800) 235-7566. 

NCTM is producing a series of booklets to support the NCTM Standards. This series 
provides instructional ideas for grades K-6 for the areas of patterns, geometry/spatial sense, 
making sense of data, and number sense/operations. They are packaged two ways: by grade 
level across all subjects, or by subject across all grade levels. It is included here because the 
exercises might be useful to suggest tasks that could be used in performance assessments. 

(TC# 500.5ADDSEK) 

Burns, Marilyn. Math and Literature (K-3), 1992. Available from: Math Solutions 

Publications, Marilyn Burns Education Associates, 150 Gate 5 Rd., Suite 101, Sausalito, 
CA 94965, (415) 332-4181, fax (415) 331-1931. 

The author's premise is that "children's books are effective classroom vehicles for motivating 
students to think and reason mathematically." This book describes instructional ideas for 
linking math and literature in grades K-3. Ten lessons are included in their entirety. Twenty- 
one additional ideas are also provided. Samples of student work are included. The book is 
included on this bibliography because it might provide ideas for tasks that would be useful in 
performance assessments. 

(TC# 500.5MATLIT) California Assessment Program. A Sampler of Mathematics 
Assessment - Addendum - Preliminary Edition, 1993. Available from: California 
Department of Education, PO Box 944272, Sacramento, CA 94244. 

The Sampler Addendum supplements a previous publication {Sampler). While the Sampler 
provided an overview of the new assessment system, the addendum provides more detail on 
the assessments, including scoring guides and scored samples of student work in grades 4, 8 
and 10 

(TC# 500.3SAMMAA2) 

California State Department of Education. A Question of Thinking: A First Look at 

Students' Performance on Open-Ended Questions in Mathematics, 1989. Available from: 
California State Department of Education, PO Box 944272, Sacramento, CA 94244- 
2720, (916) 445-1260. 

This report describes the results of 12th grade student assessment using open-ended math 
problems that were part of the California Assessment Program (CAP). The open-ended 
problems were scored using rubrics developed for each problem. These rubrics are described, 
and "anchor" papers for the six scale values for each rubric are provided. Although there is a 
separate rubric for each problem, they are all intended to reflect the following dimensions of 
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problem solving: understanding of mathematics, use of mathematical knowledge, and ability 
to communicate about mathematics. 



(TC# 500.3AQUESO) 

Carpenter, Thomas P., EKzabeth Fennema, Penelope L. Larson, et al Teachers' 

Pedagogical Content Knowledge of Students' Problem Solving in Elementary Arithmetic. 
Located in: Journal for Research in Mathematics Education 19, 1988, pp. 385-401. 

This study investigated 40 first-grade teachers' pedagogical content knowledge of children's 
solutions of addition and subtraction word problems. Most teachers could identify many of 
the critical distinctions between problems and the primary strategies that children used to 
solve different kinds of problems. But this knowledge generally was not organized into a 
coherent network that related distinctions between problems, children's solutions, and 
problem difficulty. The teachers' knowledge of whether their own students could solve 
different problems was significantly correlated with student achievement. The actual 
instruments used to assess teachers' knowledge are not included in the paper. 

(TC# 500.6TEAPEC) 

Carpenter, Thomas P., James Hiebert, Elizabeth Fennema, et al. A Framework for the 
Analysis of Teaching and Learning Understanding of Multidigit Numbers. Information 
on date and availability is unknown. 

This paper presents a way to analyze instruction in math to see whether it is designed to foster 
understanding, defined as making relevant connections between knowledge. The specific 
example in the paper relates to multidigit numbers. Dimensions of instruction thought to be 
critical in promoting understanding include such things as: the scope and sequence of 
concepts, connections among representations as a basis for establishing meaning for symbols, 
the nature of problem solving, teacher specification of solution procedures and connections, 
students' articulation of solution procedures, and coherence between and within lessons. 

Most of the paper describes each of these dimensions in detail. Several pages at the end 
discuss in general terms the kinds of tasks one could give to students to see whether they are 
making the appropriate connections. 

(TC# 500.4FRAANT) 
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Center for Talent Development. Elementary School Pre-Post Survey and Middle/High 
School Pre-Post Survey, 1992. Available from: Evaluation Coordinator, Center for 
Talent Development, Northwestern University, Andersen Hall, 2003 Sheridan Rd., 
Evanston, IL 60208, (708) 491-4979. 

This document contains surveys of student attitudes toward mathematics and science. There 
are two levels-elementary and middle/high school. It was designed for use with Access 2000 
participants who are primarily African- American and Hispanic students in an inner-city public 
school system and enrolled in a math/science/engineering enrichment program. 

No technical or contextual information is included. 

(TC# 220.3QUEELM) 

Champagne, Audrey B. Cognitive Research on Thinking in Academic Science and 

Mathematics: Implications for Practice and Policy. Located in: Enhancing Thinking 
Skills in the Sciences and Mathematics . Diane Halpern (Ed.), 1992. Available from: 
Lawrence Erlbaum Associates, Publisher, 365 Broadway, Hillsdale, NJ 07642, 
(800) 926-6579. 

Although this article is not strictly about assessment, it discusses some topics of relevance to 
assessment. Specifically, it has a very nice section on the relationship between the tasks given 
to students and what they can learn. For example, students can't learn as efficiently to 
integrate knowledge if they are never given tasks that require them to do this. This also has 
relevance to designing "authentic" tasks for performance assessments. 

(TC# 000.6COGRET) 

Charles, Randall, Frank Lester, and Phares O'Daffer. How to Evaluate Progress in Problem 
Solving, 1987. Available from: National Council of Teachers of Mathematics, 1906 
Association Dr., Reston, VA 22091. 

This monograph attempts to assist educators with the challenge of developing new techniques 
for evaluating the effectiveness of instruction in problem solving by clarifying the goals of 
problem-solving instruction, and illustrating how various evaluation techniques can be used in 
practice. Goals include: select and use problem-solving strategies, develop helpful attitudes 
and beliefs, use related knowledge, monitor and evaluate thinking while solving problems, 
solve problems in cooperative learning situations, and find correct answers. 

Evaluation strategies include: informal observation/questioning and recording results using 
anecdotal records or a checklist (two are provided); interviews (a sample interview plan is 
provided); student written or oral self-report of what's happening during a problem-solving 
experience (a list of stimulus questions is given, as is a checklist of strategies); attitude 
inventories (two are given); rating scales (three-trait analytic and focused holistic scales are 
given); and multiple-choice and completion (sample items are given to assess various problem- 
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solving abilities; many of these parallel question types mentioned by Marshall, below, to assess 
procedural and schematic knowledge). 



Many sample problems are provided. No technical information nor student sample 
performances are provided. 

(TC# 500.6HOWTOE) 

Clark, David. The Mathematics Curriculum and Teaching Program, 1988. Available from: 
Curriculum Development Centre, PO Box 34, Woden, ACT 2606, Australia. Also 
available from: ERIC ED 287 722. 

This document was developed to assist classroom teachers in grades 1 -6 to improve their day- 
to-day assessment of mathematics. Content includes: rationale for assessment alternatives in 
mathematics, instructions for a two-day inservice program using the materials, instructions on 
how classroom teachers can use the materials without training, and a series of exercises, 
formats and ideas for classroom assessment. 

Assessment ideas include: help with systematically recording information from informal 
observations using checklists and "folios" of student work, setting-up opportunities for 
assessment by giving students good tasks to do, assessing problem solving, student self- 
reflection, and communicating results. 

This is written in a very user-friendly manner and contains some good ideas, especially in the 
areas of designing tasks, problem solving and self-reflection. We found some of the 
descriptions of activities a little too sketchy. 

(TC# 500.3MCTPMA) 



Clark, Glenda. AIM High Math Identification, 1992. Austin Independent School District 
Gifted and Talented Program, 211 E. 7th St., Austin, TX 78701, (512) 476-6861. 

This set of paper-and-pencil instruments was developed as one of five indicators for placing 
grade 2-5 students in the Austin Independent School District gifted math program. Each 
instrument consists of one task to be completed individually by a student. Each task has 
multiple open-response items associated with it. Responses are scored on a scale of 1 -5 using 
task-specific scoring guides. The document includes one task per grade level, the scoring 
guide, and student work samples. Not included are technical information or contextual 
information describing what the assessments are trying to measure. 

(TOf 500.3AIMHIM) 
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Coalition of Essential Schools. [ Various Articles on Exhibitions of Mastery and Setting 
Standards], 1982-1992. Available from: Coalition of Essential Schools, Brown 
University, Box 1969, One Davol Sq., Providence, RI 02912, (401) 863-3384. 



Although not strictly about math, this series of articles discusses performance assessment 
topics and goals for students that are relevant to math. The articles are: Rethinking 
Standards ; Performances and Exhibitions: The Demonstration of Mastery ; Exhibitions: 
Facing Outward. Pointing Inward ; Steps in Planning Backwards ; Anatomy of an Exhibition : 
and The Process o f Planning Backwards . 

These articles touch on the following topics: good assessment tasks to give students, the need 
for good performance criteria, the need to have clear targets for students that are then 
translated into instruction and assessment, definition and examples of performance 
assessments, brief descriptions of some cross-disciplinary tasks, the value in planning 
performance assessments, and the notion of planning backwards (creating a vision for a Hgh 
school graduate, taking stock of current efforts to fulfill this vision, and then planning 
backward throughout K- 1 2 to make sure that we are getting students ready from the start). 

(TC# 150.6VARARD) 



Collison, Judith. Connecticut's Common Core of Learning, 1990. Available from: 
Performance Assessment Project, Connecticut Department of Education, Box 2219, 
Hartford, CT 06145, (203) 566-4001. 

The Connecticut Department of Education is developing a series of performance assessments 
in science and math. Each task has three parts: individual work to activate previous 
knowledge, group work to plan and carry out the task, and individual work to check for 
application of learning. This document provides: 

1 . A lengthy description of one of the ninth grade science tasks: "speeders." 

2. Short descriptions of 24 performance tasks in science (8 each in chemistry, physics, and 
earth sciences), and 1 8 in math. 

3 . A group discussion self-evaluation form to be used by students. 

No technical information or general scoring guides are included in this document. 
(TC# 500.3COSCII) 
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Collis, Kevin F. and Thomas A. Romberg. Collis-Romberg Mathematical Problem Solving 
Profiles, 1992. Available from: Australian Council for Educational Research Limited, 
(ACER), Private Bag 55, Camberwell Victoria 3124, Australia, (03) 277-5555, 
fax: (03)277-5500. Also available from: ASHE, PO Box 31576, Richmond, VA 23294, 
(804) 741-8991. 

This assessment device for students in grades 2 and 5 has 20 open-ended problems to solve- 
one problem in each of five areas (algebra, chance, measurement, number, and space) with 
four questions per problem area. Each question is designed to tap a developmental level of 
formal reasoning. For example, the "A" question determines whether the student can use one 
obvious piece of information from the item, while the "D" question determines whether the 
student can use an abstract general principle or hypothesis derived from the information in the 
problem. 

Responses to each question are scored right/wrong. The number of correct responses on each 
task determines a developmental level. Suggestions are given for instructional strategies for 
the various developmental levels. Technical information in the manual includes typical 
performance for various grade levels, teacher judgment on the developmental level indicated 
by each task, and additional analyses to show validity of the inferences drawn. 

(TC# 500.3COLROM) 

Commission on Standards for School Mathematics. Curriculum and Evaluation Standards 
for School Mathematics, 1989. Available from: National Council of Teachers of 
Mathematics, 1906 Association Dr., Reston, VA 22091. 

This book contains standards for curriculum and assessment that attempt to create a coherent 
vision of what it means to be mathematically literate. This book has been quoted extensively 
and appears to be the current "standard" for what should be in a math curriculum. 

The assessment section covers: three statements of philosophy concerning assessment 
(alignment, multiple sources of information, and appropriate assessment methods and uses); 
seven sections on assessing various student outcomes (e.g., problem solving, communication, 
reasoning, concepts, procedures, and dispositions); and four sections on program evaluation 
(indicators, resources, instruction, and evaluation team). Each of the seven sections on 
assessing student outcomes briefly describes what the assessment should cover and provides 
some sample assessment tasks and procedures. 

(TC# 500.5CURANE) 
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Connecticut State Department of Education. Connecticut Common Core of Learning 
Assessment, 1989-1992. Available from: Connecticut State Department of Education, 
Division of Research, Evaluation, and Assessment, 165 Capitol Ave., Room 340, 
Hartford, CT 06106, (203) 566-4001. 

This package contains a variety of documents produced between 1989 and 1992. Included 
are information about: rationale for the assessment, Connecticut's Common Core of Learning 
(student learning objectives), development process, several sample tasks and scoring 
mechanisms, student and teacher feedback forms, summaries of student and teacher feedback 
on the assessments using these forms, a process for developing performance tasks, a survey 
for student attitudes about science and mathematics, and an example of concept mapping as 
an assessment tool. 

There appear to be two kinds of tasks: complex group projects and shorter on-demand tasks 
covering individual skills. The projects attempt to get at application and extension of 
knowledge and concepts. They require some individual and some group work and extend at 
least over several days. The on-demand portion covers knowledge and it's application in 
limited situations. Performances are scored for group collaboration, process skills, and 
communication skills. Some of the rubrics are task specific and some are general; some are 
based on quantity (the number of possible solutions listed, for example) and some are more 
quality based. 

(TO 000.3CONCOC) 

Crowley, Mary L. Student Mathematics Portfolio: More Than a Display Case. Located in: 
The Mathematics Teacher 86, October 1993, pp. 544-547. 

The author discusses the use of mathematics portfolios to document progress on the "big" 
NCTM outcomes— problem solving, valuing mathematics, developing mathematical 
confidence, communicating mathematically, and reasoning mathematically. She includes a fair 
amount of detail on a middle school example—sample letter to students outlining the task, a 
description of what students need to include and a hint of how content might be assessed. 
The portfolio is semi-structured— the teacher specifies categories of entries and the student 
selects which samples of work to include in each category. Students must also write 
explanations for their selections. 

A few samples of student work are included. Detailed performance criteria are not included. 
(TC# 500.3STUMAP) 
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Csongor, Julianna E. Mirror, Mirror On The Wall.. Teaching Self-Assessment to Students. 
Located in: The Mathematics Teacher 85, November 1992, pp. 636-637. Also available 
from: Saint Maria Gosetti High School, 10th and Moore, Philadelphia, PA 19148. 

The author presents a procedure for getting high school students to self-reflect in math, 
during the final five minutes of a test, students estimate how sure they are about each answer 
they gave on the test (100%, 75%, 50%, or 0%). They can earn extra credit on the test if 
their estimates fall within 3% of their actual score. She reports that students are surprisingly 
accurate in their estimates and that the procedure works especially well with slow learners. 

(TC# 500.3MIRMIW) 

CTB McGraw-Hill. CAT/5 Performance Assessment Supplement, 1990. Available from: 
CTB/McGraw- Hill, PO Box 150, Monterey, CA 93942, (800) 538-9547, 
fax (800) 282-0266. 

The "CTB Performance Assessments" are designed to either be stand-alone or integrated with 
the CAT/5 or CTBS/4. There are five levels for grades 2-11. The total battery includes 
reading/language arts, mathematics, science, and social studies and takes 2-3 hours to 
administer. There are 12-25 short- to medium-response questions for each subtest. The math 
and science subtests take 30-40 minutes. (For the CAT/5 there is a checklist of skills that can 
be used at grades K and 1 .) 

Some questions are grouped around a common theme. Many resemble multiple-choice 
questions with the choices taken off. For example, questions on one level include: "What are 
two ways that recycling paper products helps the environment?'' "This table shows the air 
temperatures recorded every two hours from noon to midnight.. . At what time did the 
temperature shown on the thermometer most likely occur?" and "These pictures show some 
of the instruments that are used in science... List two physical properties of the water in the jar 
below that can be measured with the instruments shown in the pictures. Next to each 
property, write the name of the instrument or instruments used to measure the property." 

Some of the answers are scored right/wrong and some are scored holistically. The materials 
we received contained no examples of the holistic scoring so we are unable to describe it. 
Scoring can be done either locally or by the publisher. When the Performance Assessments 
are given with the CAT/5 or CTBS/4, results can be integrated to provide normative 
information and scores in six areas. There are only three, however that use the math and 
science subtests: demonstrating content and concept knowledge, demonstrating knowledge of 
processes/skills/procedures, and using applications/problem-solving strategies. When the 
Performance Assessments are given by themselves, only skill scores are available. 

The materials we received contain sample administration and test booklets only. No technical 
information or scoring guides are included. 

(TC# 060.3CAT5PA) 
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CTB McGraw-Hill. CTB Math Task Bank, undated. Available from: CTB/M cGraw- H ill, 
PO Box 150, Monterey, CA 93942, (800) 538-9547, fax (800) 282-0266. 



The "CTB Math Task Bank" contains 644 math tasks for grades 3-9. (There is also a 
language arts bank.) Each series of questions is comprised of one or more theme-based 
activities. For example, the ninth grade bank contains a series of questions that relate to 
"selling advertisements for the yearbook." Questions require students to do such things as: 
identify how many ads of various sizes can fit on a single page; how much money different 
types of layouts can generate; and how many ads need to be sold to cover expenses. Most 
tasks are, thus, very structured and have only one right answer. However, a few are more 
open-ended. For example, a grade 4 task is: "You and a friend are playing a guessing game. 
You think of the number 10. Your friend must guess the number you have chosen. Give your 
friend some hints to help him guess this number. On the lines below, write four different hints 
about this number using math ideas." Additionally, some tasks attempt to get at self- 
reflection: "Draw a circle around the number below that shows how easy or how hard it was 
for you to solve the problems in this task." 

The materials we received do not elaborate on scoring. It appears that most questions are 
scored right/wrong. Presumably then, the total number of points on questions covering each 
skill provide an indication of student ability. No direct judgments of problem solving, 
reasoning, communication, etc. are made. 

Tasks can be accessed by student learning goal (tasks are cross-referenced to the NCTM 
standards), theme (e.g., year book ads), or question difficulty. CTB also publishes software to 
support the task bank. This includes test generation and scoring. 

The materials we received did not contain technical information. 

(TC# 500.3CTBMAT) 



Curriculum Corporation. Mathematics— A Curriculum Profile for Australian Schools, 

Mathematics— Work Samples, and Using the Mathematics Profile, 1994 Available from: 
Curriculum Corporation, St. Nicholas PI., 141 Rathdowne St., Carlton, Victoria, 3053, 
Australia, (03) 639-0699, fax (03) 639-1616. 

These documents represent the mathematics portion of a series of publications designed to 
reconfigure instruction and assessment in Australian schools. The project, begun in 1989, was 
a joint effort by the States, Territories, and the Commonwealth of Australia, initiated by the 
Australian Education Council. 

The profiles are not performance assessments, per se, in which students are given 
predeveloped tasks. Rather, the emphasis has been on conceptualizing major student 
outcomes in each area and articulating student development toward these goals using a series 
of developmental continuums. These continuums are then used to track progress and are 
overlaid on whatever tasks and work individual teachers give to students. 
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The math profiles cover the major strands of: working mathematically, space, number, 
measurement, chance/data, and algebra. Each strand has sub-areas called "organizers." For 
example, the strand of "working mathematically" contains the organizers of: investigating, 
conjecturing, using problem-solving strategies, applying and verifying, using mathematical 
language, and working in context. Each organizer is tracked through eight levels of 
development. For example, the organizer of "using mathematical language" has "talks about 
mathematical ideas in natural language" at Level 1, and "makes fluent use of mathematical 
notation in solving problems and presenting arguments succinctly, coherently and in 
conventional forms" at Level 8. 

There are lots of support materials that describe what each strand means, how to organize 
instruction, types of activities to use with students, and how to use the profiles to track 
progress. Samples of student work are included to illustrate development. The documents 
say that the levels have been "validated," but this information is not included in the materials 
we received. 

(TC# 500.3MATCUP) 

Darling-Hammond, Linda, Lynne Einbender, Frederick Frelow, et al. Authentic 

Assessment in Practice: A Collection of Portfolios, Performance Tasks, Exhibitions, and 
Documentation, October 1993. Available from: National Center for Restructuring 
Education, Schools, and Teaching (NCREST), Box 110 Teachers College, Columbia 
University, New York, NY 10027. 

This book contains sample performance assessments for grades 1-12 in science, math, social 
studies, writing and drama from a number of sources. Formats include exhibitions, projects, 
on-demand performance assessments and portfolios. The authors have included reprints of 
papers that discuss characteristics of "authentic" assessment, performance task design, and 
portfolios. Not all assessment information is reproduced; usually the editors have excerpted 
or summarized information. Performance tasks are more thoroughly covered than 
performance criteria. In most cases no technical information or sample student responses are 
provided. 

There are four math examples for grades 1-10. Tasks include graphing, block patterns, 
building a dog pen, and doing the calculations necessary for a science experiment. 

(TC# 000.3AUTASP) 

Educational Testing Service. 1992 AP Mathematics: Free-Response Scoring Guide and 
Sample Student Answers- Calculus AH - Calculus BC. Available from: Advanced 
Placement Program, Mail Stop 85-D, Educational Testing Service, Rosedale Rd., 
Princeton, NJ 08541. 

The Advanced Placement Program ( AP) enables participating colleges to grant credit or 
appropriate placement to students who demonstrate qualifying performance on the 
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examinations. AP exams are given in art, biology, chemistry, computer science, economics, 
English, French, German, government/politics, history, Latin, mathematics, music, physics, 
psychology, and Spanish. About 50 percent of each exam is free-response (essays, problems, 
etc.). (Studio art is a portfolio assessment.) 

The sample free-response question we have is from the calculus examination. It requires 
students to show all their work. Three points are available for each part of the answer (nine 
points total). Points are awarded for the correct answer, and also for specific intermediate 
steps or products. Thus, scoring is task-specific and seems to result in a measure of 
conceptual understanding. 

The document includes the problem, the ideal solution, and scored sample student responses. 
No technical information is included. Examples are apparently available for a variety of 
advanced placement exams, however only a single calculus question is included in the 
document we have. 



(TC# 550.3ADVPLE) 



Educational Testing Service. NAEP 1992 Mathematics Report Card for the Nation and the 
States, April 1993, Report No. 23-ST02. Available from: Education Information 
Branch, Office of Educational Research and Improvement, US Department of 
Education, 555 New Jersey Ave NW, Washington, DC 20208, (800) 424-1616 or 
(202)219-1651. 

The National Assessment of Educational Progress (NAEP) is congressionally mandated. 
Tests are given in reading, writing, math, science, social studies, fine arts, writing, literature, 
career development, and computers to students in grades 4, 8, and 1 2 on a revolving basis— 
not all subject areas are given each year, this entry describes the 1992 math assessment which 
tested approximately 220,000 students from 9,000 schools. 

The on-demand tests covered numbers/operations, measurement, geometry, data 
analysis/statistics/probability, algebra/functions, and estimation. These content areas were 
covered at three levels: conceptual understanding, procedural knowledge, and problem 
solving. Some questions were multi-step. Calculators were allowed. There were both 
multiple-choice and short answer (e.g., measure an angle and write in the measurement) 
questions. Some of the questions required short explanations. For example, one fourth grade 
question asked students to choose the graph that represented the number of pockets in the 
clothing worn by a class of 20 students and provide an explanation of why they picked the 
graph they did. Responses were scored right/wrong. 

The 1992 math assessment marks a shift to reporting by proficiency levels. For example, the 
"Basic" level designates conceptual and procedural understanding, while the "Advanced" level 
represents the ability to generalize and synthesize concepts and principles. (Note: The setting 
of cut-off scores on the tests relative to these levels has been controversial from a technical 
standpoint and will undergo further refinement.) 
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The report contains NAEP background information on the achievement levels and how they 
were set, sample items illustrating the levels, and lots of tables of results by states, regions, 
and various demographics such as gender and ethnicity. Released sets of items are available in 
other documents. 

(TC# 500.6NAEPMAR) 

EQUALS. Assessment Alternatives in Mathematics, 1989. Available from: University of 
California, Lawrence Hall of Science, Berkeley, CA 94720, (415) 642-1823. 

This document provides an overview of some possible assessment, methods in mathematics 
that cover both process and products. Specific examples are provided for writing in 
mathematics, mathematical investigations, open-ended questions, performance assessment, 
observations, interviews, and student self-assessment. Any of the student-generated material 
could be self-selected for a portfolio of work. The document also includes a discussion of 
assessment issues and a list of probing questions teachers can use during instruction. 

(TC# 500.6ASSALI) 

Ferguson, Shelly. Zeroing in on Math Abilities, 1992. Located in: Learn in292 . Vol. 21, 
pp. 38-41. 

The paper was written by a fourth grade teacher and describes her use of portfolios in math- 
what she has students put in their portfolios, the role of self-reflection, getting parents 
involved, and grading. She gives a lot of practical help. One interesting idea in the paper has 
to do with grading. At the end of the grading period she reviews the portfolios for attainment 
of concepts taught (not amount of work done), and progress toward six goals set by the 
NCTM standards (e.g., thinks mathematically, communicates mathematically, and uses tools). 
She marks which goals were illustrated by the various pieces of work in the portfolio and 
writes a narrative to the student. 

Another interesting idea is formal presentations of their portfolios by students to their parents. 
The article provides a sample comment form for parents and students to complete. 

(TC# 500.3ZERMAA) 



Finston, D., A. Knoebel, and D. Kurtz. Student Assessment Using Student Research 
Projects, 1993. Available from: Douglas S. Kurtz, Professor of Mathematics, 
Department of Mathematical Sciences, New Mexico State University, Box 30001, 
Las Cruces, NM 88003, (505) 646-6218. 

This three-year-old prototype program (used in grades 9-university) is based on "student 
research projects," multi-step assignments lasting up to several weeks, which attempt to evoke 
diverse problem-solving skills and well-written solutions in grammatically correct prose. In 
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the course of completing a project, students read, write and speak mathematics. Students 
work in groups of 3-4. Tasks require open-ended, extended responses. Samples in the 
document include remodeling a house, deciphering averages, determining tunnel clearances, 
and designing revolutionary solids. 

Assessment of student work is based on teacher evaluations of written reports. Work may be 
graded holistically or based on predetermined guidelines. The sample scoring guides included 
in the document generally include content, process, and communication skills, but are fairly 
sketchy. 

No technical information is included. 

The author is in the process of preparing a book compiling information on the student 
research projects, but grants permission to educators for reproduction. 

(TC# 500.3STUASU) 

Fitzpatrick, Anne R., Kadriye Ercikan, and Steven Ferrara. An Analysis of the Technical 
Characteristics of Scoring Rules for Constructed-Response Items, 1992. Available from: 
CTB Macmillan/McGraw-Hill, PO Box 150, Monterey, CA 93942-0150, (800) 538-9547. 

This was a paper presented at the annual meeting of the National Council on Measurement in 
Education, San Francisco, April 1992. 

This paper reports on a technical study of the open-response portion of the 1991 
administration of the Maryland state tests in reading and math. Items had a variety of scoring 
formats including different number of possible points and scoring tied to individual tasks. 
Results showed that the math open-response questions were hard, discriminated well between 
students having different achievement levels, and worked better when more score points were 
used. Thus, there is evidence that this set of open-response questions might offer more 
measurement accuracy than multiple-choice questions. 

(TC# 060.6ANATEC) 

Fitzpatrick, Robert and Edward J. Morrison. Performance and Product Evaluation. 
Located in: Educational Performance Assessment , Fredrick L. Finch (Ed.), 1991. 
Available from: The Riverside Publishing Company, 8420 Bryn Mawr Ave., Chicago, 
1L 60631, (800) 323-9540. 

This paper has interesting discussions of the following topics: 

1 . What "authenticity" in tasks means. The authors' position is that there are many degrees 
and kinds of artificialities in tests. "Performance and product evaluation are those in 
which some criterion situation is simulated to a much greater degree than is represented 
by the usual paper-and-pencil test... [However,] there is no absolute distinction 
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between performance tests and other classes of tests—the performance test is one that is 
relatively realistic. " 

2. Criteria for deciding how much "reality" to include in tasks. 

3. Descriptions of various types of tasks that can be used in performance assessments: 
in-basket, games, role-plays, projects, etc. 

4. Steps for developing performance assessments: analysis of the important dimensions of 
the skills to be covered, identification of tasks that cover as many of the important skills 
as possible, developing instructions and materials, and developing the scoring procedure. 

Most specific examples are taken from military and business applications. 

(TC# 150.6PERPRE) 

Fraser, Barry J., John A. M alone, and Jillian M. Neale. Assessing and Improving the 
Psychosocial Environment of Mathematics Classrooms. Located in: Journal for 
Research in Mathematics Education 20, 1989, pp. 191-201. 

This article describes the development of a short form of the My ( lass Inventory to be used in 
sixth grade math classes to measure the psychosocial characteristics of the classroom learning 
environment, i.e., social interactions. 

(TC# 500.3ASSIMP) 

Glaser, Robert. Expert Knowledge and Processes of Thinking. Located in: Enhancing 
Thinking Skills in the Sciences and Mathematics , Diane Halpern (Ed.), 1992. Available 
from: Lawrence Erlbaum Associates, Publisher, 365 Broadway, Hillsdale, NJ 07642, 
(800) 926-6579. 

In this article the author describes research on expert performance. Although not directly 
about assessment, expeit performance can be used to help understand and define the targets 
we have for students, which is the first step toward designing assessment. For example, 
expert performance can be used to develop criteria for evaluating performance tasks. 

The author points out that, although expertise is very subject-specific, generalizations can be 
made about its nature across subjects: experts perceive large, meaningful patterns, have 
skillful self-regulatory processes, etc. 

A critical point made by the author is that, "Practice, as it comes about in the usual course of 
training, is not necessarily very efficient. On the basis of our knowledge of the specific 
aspects of competence and expertise, we are able to find ways to compress or shortcut 
experience. .. " "This is one goal for performance assessment, we help students understand 
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current conceptions of the relevant dimensions of a task so that they don't have to rediscover 
this themselves. ' 

(TC# 050.6EXPKNP) 

Greenwood, Jonathan Jay. On the Nature of Teaching and Assessing "Mathematical 
Power" and "Mathematical Thinking. " Located in: Arithmetic Teacher , November 
1993, pp. 144-152. 

The authors discuss the following topics: (1 ) the meaning of the NCTM goals of 
"mathematical power" and "mathematical thinking," (2) seven characteristics of sound 
mathematical thinking and ideas for instruction, and (3) general rating scales for rating each 
characteristic. 

The seven characteristics are: (1) Everything you do in mathematics should make sense to 
you. (2) Whenever you get stuck, you should be able to use what you know to get yourself 
unstuck. (3) ^ou should be able to identify errors in answers, in the use of materials, and in 
thinking. (4) Whenever you do a computation, you should use a minimum of counting. 
(5) You should be able to perform calculations with a minimum of rote pencil-paper 
computations. (6) When the strategy you are using isn't working, you should be willing to try 
another strategy intead of giving up. (7) You should be able to extend, or change, a problem 
situation by posing additional conditions or questions. 

This is a useful paper for assisting teachers in knowing the characteristics of sound 
mathematical thinking. 

No technical information is included. Sample instructional tasks and student performances are 
included. 

(TC# 500.6ONNATT) 

Hall, Greg. Alberta Grade 9 Performance-Based Assessment— Math, 1992. Available from: 
Greg Hall, Student Evaluation Branch, Alberta Education, Box 43, 11160 Jasper Ave., 
Edmonton, AB T5K 0L2, Canada. 

The 1 992 ninth grade math performance assessment entailed six stations with hands-on 
activities. Students circulate through the stations; testing time for each group of six students 
is 90 minutes. Some of the six tasks were open-response and some were open-ended; all were 
assessed for problem solving. The six tasks involved applications of rearranging squares to 
form different perimeters for the same area, measurement and mapping, surface area, 
collecting and graphing information, estimation, and combinations/permutations. (The Grade 
9 pilot field-tested 14 other tasks. The Grade 6 pilot field-tested 1 1 tasks.) 

Responses to the Grade 9 tasks were scored using an analytical trait system having two 
dimensions: problem solving and communication. Each trait was scored on a scale of 0 

NWREL, August 1 994 21 Mathematics 

Test Center, (503) 275-9582 2 "I 



(totally misunderstood or blank) to 3 (readily understood the task, developed a good strategy, 
carried out the strategy and generalized tne conclusion). A few possible student responses are 
included to illustrate scoring, but no actual student responses are included. No technical 
information is included. 

(TC# 500.3 ALBGRN) 

Halpern, Diane (Ed.). Enhancing Thinking Skills in the Sciences and in Mathematics, 1992. 
Available from: Lawrence Erlbaum Associates, Publishers, 365 Broadway, Hillsdale, 
NJ 07642, (800) 926-6579. 

This book is not strictly about assessment. Rather, it discusses the related topics of "What 
should we teach students to do?" and "How do we do it?" The seven authors "criticize the 
conventional approach to teaching science and math, which emphasizes the transmission of 
factual information and rote procedures applied to inappropriate problems, allows little 
opportunity for students to engage in scientific or mathematical thinking, and produces inert 
knowledge and thinking skills limited to a narrow range of academic problems." (p. 118). In 
general, they recommend that teachers focus on the knowledge structures that students should 
know, use real tasks, and set up instruction that requires active intellectual engagement. 

The authors give various suggestions on how to bring this about: instructional methods, 
videodiscs, group work, and a host more. The final chapter analyzes the various positions and 
raises theoretical issues. 

(TC# 500.6ENHTHS) 

Hartman, Charlotte. Mathematical Power Opens Doors, 1993. Available from: Vancouver 
School District, PO Box 8937, Vancouver, WA 98668, (206) 696-701 1. 

Vancouver School District is developing open-ended mathematics problems to supplement its 
multiple-choice tests of content. Pilots are being undertaken in several grade levels; however, 
we only have examples for grade 6. 

The document we have contains: ( 1 ) a restatement of the "big" NCTM outcomes (problem 
solving, reasoning, communication, connections); (2) three sample problems (all written and 
completed individually); (3) a scoring guide (generalized scoring guide rating the five traits of 
problem solving, communication, reasoning, math concepts and math procedures). The 
district is gathering benchmarks to illustrate each of the five traits. 

A personal communication indicated that materials are used in two phases. First, several 
problems are done in a group to model how to proceed. Then, students choose three of five 
problems to do individually. 
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No technical information is available. 



(TC# 500.3MATPOO) 

Harvey, John G, Mathematics Testing With Calculators: Ransoming the Hostages. Located 
in: Mathematics Assessment and Evaluation: Imperatives for Mathematics Educators , 
Thomas A. Romberg (Ed.), 1992. Available from: State University of New York Press, 
State University Plaza, Albany, NY 12246. 

This paper looks at the use of calculators in mathematics testing. The premise is that if we 
want students to investigate, explore and discover, assessment must not just measure mimicry 
math. Tests designed to really require calculators are more likely to be able to do this. 
Additionally, it is important to incorporate calculators into the curriculum because in the 
technological world of the future, calculators will be essential. If we want teachers to use 
calculators in instruction, we need to incorporate them into testing. 

The author analyzes three types of test with respect to calculator use, describes things to 
consider when designing calculator tests, and describes current activity in developing 
"calculator-active" tests. 

(TC# 500.6MATTEC) 

Hibbard, K. Michael. Region IS - Together for Students - A Community of Learners, 1993. 
Available from: Region 15 School District, PO Box 395, Middlebury, CT 06762, 
(203) 758-8250. 

Pomperaug Regional School District 15 staff have worked for seven years to define the most 
important products (such as graphs) and performances (such as oral presentations) which 
students should be able to make and do. "Lists" (rubrics), developed for each skill at the 
elementary, middle and high school levels, describe the salient features of performance and are 
used for grading and student self-assessment. 

This document contains handouts used in a training session that appear to be an overview of 
the district student assessment system, in addition to a general overview and philosophy 
statement, the handouts include sample assessment materials in science, social studies, math, 
and writing for grades 1-12. 

The math information includes checklists for assessing graphs and two sample graphing 
activities. 

No technical information nor samples of student work are included. 
(TC# 000.6TOGSTC) 

o 
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Hibbrrd, K. Michael. Self-As^ intent Using Performance Task Assessment Lists, 1994. 
Available from: Region 15 School District, PO Box 395, Middlebury, CT 06762, 
(203) 758-8250. 

Pomperaug Regional School District 15 (Middlebury, CT) has developed performance criteria 
for a wide variety of student products and performances. These criteria, called "task 
assessment lists" are written to be developmentally appropriate for various grade levels 
(primary, elementary, and secondary). The lists are used to communicate to students the 
attributes of quality products and performances, promote student self-assessment and self- 
reflection, and grading. This document contains sample task assessment lists for graphs. 

(TC# 500.3SELUSP) 

Horn, Kermit, and Marilyn Olson. 1992-1993 Lane County Fourth Annual Project Fair. 
Official Guidelines, Criteria & Registration Forms for Grades K-12. Available from: 
Kermit Horn or Marilyn Olson, Project Fair Coordinators, Lane Education Service 
District, Instructional Services Division, PO Box 2680, Eugene, OR 97402, 
(503) 689-6500. 

This document is the handbook given to students in grades K-12 interested in registering for 
the Lane County project fair. It contains information on registration, criteria by which 
projects will be judged, as well as help with getting started. 

The document also gives some excellent ideas on interdisciplinary projects. 

Some journal entries from past submissions are included to show students what to do No 
samples that illustrate score points on criteria are included; the criteria, although an excellent 
start, are still a little sketchy. 

(TC# 000.3LANCOP) 

Hynes, Michael C. K-5 Mathematics Program Evaluation - A Florida Model Curriculum 
Project; School Handbook, 1993. Available from: University of Central Florida, College 
of Education, Orlando, FL 32816, (407) 823-6076. 

In cooperation with a team of educators throughout the state of Florida, the author has 
produced a handbook intended to provide schools with the tools to assess the K-5 
mathematics program dimensions of: student outcomes, program goals, curriculum, learning 
environment, learning resources, program support, evaluation, and equal access. The 
handbook includes sample instruments for each area. Most of these are surveys except for 
assessment of student outcomes. 

Seventeen sample performance tasks (eight for primary and nine for grade 5) are included to 
assess various student outcome goals. These are open-ended (there is more than one correct 
answer) and most require explanations of, and rationale for, procedures used. Tasks do not 
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require manipulatives; all are written/visual. All tasks are done individually; there is no group 
collaboration. Performance is rated using a generic four-point scale--"Exemplary (Level A)" 
to "No Attempt (Level E) " An Exemplary response is one which: "Provides excellent 
responses in all areas of problem solving, reasoning, communication, connections, and 
mathematics content. Goes beyond the excellent category. Shows creative thinking, elegant 
communication, and/or unique approaches to the problem. Uses additional information to 
extend the solution." A sample student response for each score point is included to illustrate 
the scoring procedure. 

A sample "School Report Card" is included to illustrate "a succinct way to report the results" 
of all surveys and assessments. 

The handbook includes a disk containing each of the instruments, the "School Report Card" 
and the scoring rubric. No technical information is included. The author has granted 
permission for educators to reproduce materials for use with students. 

(TC# 500.3K-5MAP) 

Illinois State Board of Education. Defining and Setting Standards for the Illinois Goal 
Assessment Program, (IGAP), 1991. Available from: Illinois State Board of Education, 
100 N. 1st St., Springfield, IL 62777. 

This paper describes Illinois' procedure for setting standards on the IGAP in grades 3, 6, 8, 
and 1 1 . The steps include: 

1 . Creating descriptions of what students look like at three levels of competence: does not 
meet the state goal for learning, meets the state goal for learning, and exceeds the state 
goal for learning 

2. Judgments by educators of the percent of students at each level that are likely to get 
each item correct 

3. Adjustment of judgments by looking at the actual percentage of students getting the 
items correct 

The paper includes a description of the process and descriptions of students at grades 3, 6, 8, 
and 1 1 at each level of competence in math. 

(TC# 000.6DEFSES) 

Johnson, Judi Mathis. Portfolio Assessment in Mathematics: Lessons from the Field. 
Located in: The Computino Teacher 21, March 1994, pp. 22-23. 

The author describes the results of a study in which she interviewed 20 successful math 
teachers. Her major conclusion is that math, assessment, and technology should not be 
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competitors for teachers' attention, but components of a successful math classroom. 
Portfolios are used to document student learning and promote learning and student 
responsibility for learning through self-reflection. Technology is used to learn concepts, 
produce work for the portfolios, and to produce the portfolio itself. Assessment is used to 
guide planning and influence instruction. 

(TC# 500.6PORMAL) 

Kansas State Board of Education. Kansas Mathematics Standards and 1991 Kansas 
Statewide Pilot Assessment Results, 1991. Available from: Kansas State Board of 
Education, Kansas State Education Building, 120 SE 10th Ave., Topeka, KS 66612. 

This is an overview of the 1991 Kansas pilot math assessment and a description of results. 
Students from grades 3, 7, and 10 were tested. The pilot included both multiple-choice and 
open-performance problems. The performance assessment portion entailed giving 1/6 of the 
students tested one task each. A total of 3 1 tasks were used altogether in the three grades. 
Nine problems are included in the report. 

Responses were scored using both a holistic scale (0-6) for overall correctness of response, 
and a four-trait analytic model focusing on problem-solving processes (understanding the 
question, planning, implementing the strategies selected, and verifying the results). Each trait 
is rated on a six-point scale (A-F). Scoring guides are included, but detailed instructions and 
sample student work are not. 

Some information on student performance is included, but no other technical information on 
the test itself is included. 

(TC# 500.3KASMAS) 

Katims, Nancy, Pat Nash, and Cynthia M. Tocci. Linking Instruction and Assessment in a 
Middle School Mathematics Classroom. Located in: Middle School Journal 25, 
November 1993, pp. 28-35. 

Educational Testing Service (ETS) is developing the PACKETS Program, consisting of math 
exercises designed to promote the "big ideas" in math through activities relevant to students. 
PACKETS activities, for grades 6-8, pose real-life problems for which the solution must 
address the needs of a specified client. Some group and some individual work is required. 

The article includes a few sample exercises, sample lesson plans, teacher and student 
responses to the tasks, and sample student work. It appears that scoring is left up to teachers. 
Work is assessed on the dimensions of problem-solving strategy, communication, math 
concepts and completeness of the answei . Criteria for assessing work and student 
understanding of the criteria are only sketchily discussed. 

(TC# 500.3LIN1NA) 
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Kloosterman, Peter, and Frances K. Stage. Measuring Beliefs About Mathematical Problem 
Solving. Located in: School Science and Mathematics 92, March 1992, pp. 109-115. 



The authors describe the development of a scale to assess student beliefs about mathematics. 
The instrument, designed for grades 7+, is called the Indiana Mathematics Beliefs Scale. 
Thirty-six questions cover six beliefs. (1)1 can solve time-consuming mathematics problems. 

(2) There are word problems that cannot be solved with simple, step-by-step procedures. 

(3) Understanding concepts is important in mathematics. (4) Word problems are important in 
mathematics. (5) Effort can increase mathematical ability. (6) Mathematics is useful in daily 
life 

The paper includes technical information based on studies with college students. 
(TC# 500.3MEABEM) 



Knight, Pam. How I Use Portfolios in Mathematics, 1992. Located in: Educational 
Leadership 49, pp. 71-72. Also available from: Twin Peaks Middle School, Poway 
Unified School District, 14012 Valley Springs Road, Poway, CA 92064. 

The author describes her first year experimentation with portfolios in her middle school 
algebra classes. She had her students keep all their work for a period of time and then sort 
through it to pick entries that would best show their effort and learning in algebra and the 
activities that had been the most meaningful. There is some help with what she did to get 
started and discussion of the positive effects on students. There is some mention of 
performance criteria, but no elaboration. One student self-reflection is included, but no 
technical information. 

(TC# 530.3HOWIUS) 



Koretz, Daniel, Daniel McCaffrey, Stephen Klein, et al. The Reliability of Scores from the 
1992 Vermont Portfolio Assessment Program— Interim Report, December 1992. Available 
from: RAND Institute on Education and Training, National Center for Research on 
Evaluation, Standards, and Student Testing, UCLA Graduate School of Education, 
10880 Wilshire Blvd., Los Angeles, CA 90024, (310) 206-1532. 

Beginning in 1990, RAND has been carrying out a multi-faceted evaluation of Vermont's 
portfolio assessment program. This paper reports on reliability findings of the study 
conducted during school year 1991-92. Basically, RAND found that interrater agreement on 
portfolio scores was very low for both writing and math. The authors speculate that this 
resulted from aspects of scoring systems, aspects of the operation of the program, and the 
nature and extent of training raters. 
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This report provides good advise and caution for others setting up portfolio systems for large- 
scale assessment. 

(TC# 150.6RELSCV) 

Koretz, Daniel, Brian Stecher, and Edward Deibert. The Vermont Portfolio Assessment 
Program: Interim Report on Implementation and Impact, 1991-92 School Year, 
Available from: RAND Institute on Education and Training, National Center for 
Research on Evaluation, Standards, and Student Testing, UCLA Graduate School of 
Education, 10880 Wilshire Blvd., Los Angeles, CA 90024, (310) 206-1532. 

Beginning in 1990, RAND has been carrying out a multi-faceted evaluation of Vermont's 
portfolio assessment program. This paper reports on questionnaires and interviews conducted 
during school years 1990-91 and 1991-92. Results indicated that: 

1. There was a significant impact on instruction, but teachers felt somewhat confused about 
what they were supposed to do. 

2. The portfolios took a lot of classroom space and tended to be viewed by teachers as an 
add-on rather than as "the" instruction. 

3. Teachers felt they knew more about students as the result of doing portfolios. 

4. Students had some difficulty doing portfolio problems. 

5. Reported effect on low achieving students was mixed. 
(TO 150.*VERPOP) 

Kulm, Gerald. (Ed.) Assessing Higher Order Thinking in Mathematics, 1990. Available 
from: American Association for the Advancement of Science, 1333 H Street NW, 
Washington, DC 20005, (301) 645-5643. 

This book contains a series of articles that address various topics in mathematics assessment. 
The articles address three broad topics: 

1 . The rationale for assessing mathematics problem solving and the need to have 
assessment devices that reflect this emphasis. 

2. Issues that come up when trying to assess higher-order thinking skills in mathematics. 

3. General discussions of what to assess and how to assess it. 
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There are a few examples of actual assessment techniques. The most relevant articles are 
included on this bibliography as separate entries. 



(TC# 500.6ASSHIO) 



Lambdin, Diana V. and Vicki L. Walker. Planning for Classroom Portfolio Assessment. 
Located in: Arithmetic Teacher , February 1994, pp. 318-324. 

The authors report on the development of a portfolio system for their secondary mathematics 
students. (This was part of a larger project for all grade 3-12 teachers.) 

The authors have students assemble a "permanent portfolio" from a "working portfolio." The 
working portfolio is a place to store all potential permanent portfolio entries. The permanent 
portfolio is assembled from the working portfolio. All portfolios are stored in an easily 
accessible place in the classroom. The goals are to promote student self-assessment, improve 
communication with students and parents, and gain a broader picture of the student than 
available with traditional tests. The goal is not to assess student knowledge. The authors 
include task guidelines that are given to students. 

In addition to student and peer evaluation of portfolio selections and the portfolio as a whole, 
the teacher has guidelines for grading. Grades are based on diversity of selection, written 
reflections about selections, and portfolio organization. The authors present a little detail that 
flesh out these criteria. 

The document includes some student work but no technical information. 
(TO 500.3PLAPOA) 

Lane, Suzanne. QUASAR Cognitive Assessment Instrument, (QCAI), 1993. Available from: 
QUASAR (Quantitative Understanding: Amplifying Student Achievement and 
Reasoning), Learning Research & Development Center, University of Pittsburgh, 
3939 O'Hara St., Pittsburgh, PA 15260, (412) 624-7791. 

The QCAI (QUASAR Cognitive Assessment Instrument) is designed to measure long-term 
growth of students in the area of math thinking and reasoning skills. Information for this 
review was taken from the following publications: Principles for Developing Performance 
Assessments: An Example of Their Implementation (Lane & Carol Parke, AERA, 1992); 
Empirical Evidence for tie Reliability and Validity of Performance Assessments (Lane, 
Clement Stone, Robert Ankenmann & Mai Liu, AERA, 1992); The Conceptual Framework 
for Development of a Mathematics Performance Assessment Instrument (Lane, AERA, 
1991 ); Validity Evidence for Cognitive Complexity of Performance Assessments: An 
Analysis of Selected QUASAR Tasks (Maria Magone, Jinfa Cai, Edward Silver, and Nign 
Wang, AERA, 1992); and Conceptual and Operational Aspects of Rating Student Responses 
to Performance Assessments (Patricia Kenney and Huixing Tang, AERA, 1992). 
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Thirty-three tasks were designed for sixth and seventh graders. No single student receives 
more than nine tasks in any 45-minute sitting. The tasks were designed to provide a good 
sample of math thinking and reasoning skills by having a variety of representations, 
approaches and problem strategies. Specifically, students were asked to provide a justification 
for a selected answer or strategy, explain or show how an answer was found, translate a 
problem into another representation (picture or equation), pose a mathematical question, 
interpret provided data, and extend a pattern and describe underlying regularities. The tasks 
were carefully field-tested for bias and confusing or difficult instructions. General descriptions 
for all the tasks, and details on a few individual tasks, are provided in these materials. 

Scoring is done via a generalized holistic, four-point rubric which directs raters to consider 
mathematical knowledge, strategic knowledge and communication. (Each of these dimensions 
is laid out very clearly and could be used as the basis of an analytical trait scoring scale.) The 
generalized rubric is then applied to each problem by specifying features of responses that 
would fall at different scale points. The generalized scoring guide is included in these 
materials, but not the task-specific adaptations. 

(TC# 500.3QUACOA) 

Larter, Sylvia. Benchmarks: The Development of a New Approach to Student Evaluation, 
1991. Available from: Toronto Board of Education, 155 College St., Toronto, 
ON M5T 1P6, Canada, (416) 598-4931. 

Benchmarks are student performances on tasks tied to Provincial educational goals. Each 
Benchmark activity lists the goals to be addressed, the task, and the scoring system. To 
develop the Benchmarks, two observers were used for each student-one to interact with the 
student and one to record observations. Tasks vary considerably. Some require very discrete 
responses (e.g., knowledge of multiplication facts using whatever means the student needs to 
complete the task), while some are more open-ended. There are 129 Benchmarks developed 
in language and mathematics for grades 3, 6, and 8. 

For many of the tasks, a general, holistic, seven-point scale ("no response" to "exceptional 
performance [rare]") was used as the basis to develop five-point, task-specific, holistic scoring 
scales. For other tasks, scoring appears to be right/wrong. Holistic scoring seems to 
emphasize problem solving, method of production, process skills, and accuracy, although 
students can also be rated on perseverance, confidence, willingness, and prior knowledge, 
depending on the Benchmark. 

The percentage of students at each score point (e.g., 1-5) is given for comparison purposes, as 
are other statistics (such as norms) when appropriate. Anchor performances (e.g., what a "3" 
performance looks like) are available either on video or in hard copy. 
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This report describes the philosophy behind Benchmarks, how they were developed, and a few 
of the specific Benchmarks. Some technical information is described (factor analysis, rater 
agreement), but no student performances are provided. 

(TC# 100.6BENCHM) 

Lash, Andrea. Arithmetic Word Problems: Activities to Engage Students in Problem 

Analysis, 1985. Available from: Far West Laboratory, 730 Harrison St., San Francisco, 
CA 94107, (415) 565-3000. 

This is a book of arithmetic word problems selected by the author to promote problem 
solving. Some are multiple-choice and some are open-response. The author categorizes 
problems as being "word problems," "process problems," "applied problems," and "puzzle 
problems." The author also presents a model for the steps in problem solving and a discussion 
of the implications for instruction. Problems are grouped according to the step in the 
problem-solving process they relate to. 

Most of the problems have only one right answer and none seem to utilize manipulatives. 
However, problems are presented for addition, subtraction, multiplication, division, multi-step 
problems, and problems containing unnecessary information. Some might be useful as tasks in 
a performance assessment. 

(TC# 500.2ARIWOP) 

Lawrence, Barbara. Utah Core Curriculum Performance Assessment Program: 
Mathematics, 1993. Available from: Profiles Corporation, 507 Highland Ave., 
Iowa City, IA 52240. 

The Utah State Office of Education has developed 90 constructed-response items in 
mathematics, science and social studies (five in each of grades 1 -6 for each subject) to 
complement multiple-choice tests already in place. Assessments are designed to match the 
Utah Core Curriculum goals. Although districts are required to assess progress toward these 
goals, the state-developed assessments are optional. 

The mathematics assessments are designed to measure six to ten of the following areas of 
student competence (depending on grade level): logical reasoning, number meanings, number 
operations, number representation, computation, estimation, algebra, data sets, probability, 
geometry, measurement, fractions, and patterns. Each task has several questions relating to 
the same theme. For example, a sixth grade task called "Lab Tech" has students do such 
things as: complete a number sequence (adding milliliters to a chemical solution every 10 
minutes), and "You need to plant two kinds of seeds. You must have 1 2 pots of one kind of 
seeds and 1 8 pots of the other kind of seeds. You need to plant the same total number of 
each kind of seed. What is the least number of each kind of seed you could plant?" 
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Scoring is task-specific and based on the degree of correctness of the response. For example, 
in the "Lab Tech" example, the student gets 3 points if he or she correctly completes the 
entire sequence. Points are totaled for each task and between tasks for each of the skill areas 
being assessed. Four levels of proficiency on each skill are identified: advanced, proficient, 
basic and below basic. Cut scores for each level are based on percent correct (approximately 
90%=advanced, 70%=proficient, 40%=--basic, below 40%=below basic) and behavioral 
descriptions of performance at each level. 

Assessment activities are bound in books for each grade level/subject. Each task includes 
teacher directions, student test-taking materials, and scoring guides. The Office of Education 
has collected information on teacher reaction to the assessments from the field test No other 
technical information is available at this time. A training video is available which helps 
teachers use the assessments. 

(TC# 500.3UTACOC and 000.6INTUTCv~vtt/eo) 

Leach, Eilene L. An Alternative Form of Evaluation that Complies with NCTM's Standards. 
Located in: The Mathematics Teacher 85, November 1992, pp. 628-632. Also available 
from Centaurus High School, 10300 S. Boulder Rd., Lafayette, CO 80026. 

This teacher uses scored discussions to assess and promote problem solving, communicating 
mathematically, and group process skills in her high school math classes. She has three to six 
students face each other in front of the rest of the class and spend about five minutes trying to 
solve a problem. Individuals can earn positive points for such things as "determining a 
possible strategy to use," "recognizing misused properties or arithmetic errors," or "moving 
the discussion along." They can earn negative points by doing such things as: "not paying 
attention or distracting others," and "monopolizing." 

The article has a thorough discussion of how the teacher sets up the classroom, introduces the 
procedure to students, scores the discussion, and handles logistics. The author also discusses 
the positive effects this procedure has had on students, and the additional insight she has 
obtained about her students. 

All scoring is teacher-centered, but it wouldn't necessarily have to be. No technical 
information is included. 

(TC# 500.3ALTFOE) 

Lehman, Michael. Assessing Assessment: Investigating a Mathematics Performance 
Assessment, 1992. Available from: The National Center for Research on Teacher 
Learning, 1 16 Erickson Hall, Michigan State University, East Lansing, MI 48824-1034. 

This monograph, by a high school math teacher, describes his attempt to develop a better 
method of assessing algebra problem solving, concepts, and skills than traditional paper-and- 
pencil tests. The assessment technique involves giving students problems to solve as a group. 
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and then having them explain their results in front of a panel of judges. Three examples of 
problems are provided, as is a brief description of the scoring criteria (making sense of the 
problem, and problem-solving strategies), accuracy of results, interpreting results, ability to 
communicate results, and an explanation of what they did. However, these criteria are not 
elaborated on, and, although samples of student explanations are provided, these are used to 
describe the understandings the teacher reached about his students, not to anchor the 
performance criteria. 

The author also provides a brief summary of the strategies he uses to help students develop 
greater depth in their understanding of algebraic principals and their interrelationships— small 
group cooperative learning, requiring justifications of approaches, etc. 

(TC# 530.3ASSASI) 

Lehman, Michael. Performance Assessment— Math, 1992. Available from: Michael 

Lehman, Holt Senior High School, 1784 Aurelius Rd., Holt, MI 48842, (517) 694-2162. 

This paper is related to the one above, and provides additional information. Students are 
given six problems (some having only one right answer and some having more than one right 
answer) to solve as a team (four students per team). The team then spends an hour with a 
panel of three judges. Judges can ask any student to explain the team's solution and problem- 
solving strategy on any of the six problems. (Therefore, all students must have knowledge of 
all six problems.) Then the judges assign the team a new problem to work on while they 
watch. 

Student responses are scored on: making sense of the problem, solution strategies, accuracy 
of results, ability to communicate results, ability to answer questions posed by the judges, 
three judgments of group-process skills, and an overall judgment of student understanding. 

A complete set of ten tasks (six pre-assigned, and four on-the-spot) are included for 
Algebra II. The scoring guide and a few sample pre-calculus projects are also included. No 
technical information nor sample student performances are included. 

(TC# 500.3PERASM) 

Lesh, Richard. Computer-Based Assessment of Higher Order Understandings and Processes 
in Elementary Mathematics. Located in: Assessing Higher Order Thinking in 
Mathematics , Gerald Kulm (Ed.), 1990. Available from: American Association for the 
Advancement of Science, 1333 H Street NW, Washington, DC 20005, (301) 645-5643. 

This article is as much about how meaningful learning occurs and the nature of the structure 
of knowledge in mathematics, as it is about use of computers in math instruction and 
assessment. The basic premise is that computer-based tests should not simply be pencil-and- 
paper tests delivered on-line. They should be part of an integrated instruction and assessment 
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system that supports both learning facts and developing the meaningful internal structuring of 
these facts to form a coherent knowledge system. 



The article discusses three things: 

1. Principles underlying a modeling perspective of learning and assessment (ideas such as: 
learning and problem-solving situations are interpreted by the learner by mapping them 
to internal models, and several "correct" alternative models may be available to interpret 
a given situation) 

2. Five objectives that should be emphasized in K-12 math (such as going beyond isolated 
bits of knowledge to construct well-organized systems of knowledge, and think about 
thinking) 

3. Specific types of assessment items that can be used to measure these deeper and broader 
understandings (such as conceptual networks and interactive word problems) 

Many sample problems are provided. 

(TC# 500.6COMBAA) 

Lesh, Richard and Susan J. Lamon, Eds. Assessment of Authentic Performance in School 
Mathematics, 1992. Available from: American Association for the Advancement of 
Science, 1333 H St. NW, Washington, DC 20005. 

This book contains articles by several authors on the topics of: assessment objectives-what 
should we assess; examples of assessments; classroom assessment; and scoring and reporting. 

(TC# 500.6ASSAUP) 



Lester, Frank K, Jr. An Assessment Model for Mathematical Problem Solving. Located in: 
Teaching Thinking and Problem Solving 10, September/October, 1988, pp. 4-7. Also 
available from: Lawrence Erlbaum Associates, Inc., Journal Subscription Department, 
365 Broadway, Hillsdale, NJ 07642, (800) 962-6579 

This article presents a model for assessing both the problem-solving performance of students 
and assessing the task demands of the problem to be solved. The dimensions of problem 
solving (which could be used as a scoring rubric) are: understanding/formulating the question 
in a probiem, understanding the conditions and variables in the problem, selecting the data 
needed to solve the problem, formulating subgoals and selecting appropriate solution 
strategies to pursue, implementing the solution strategy and attaining subgoals, providing an 
answer in terms of the data in the problem, and evaluating the reasonableness of an answer. 
The article describes these in some detail. 
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The problem features th?t can affect a student's success in solving a problem are: the type of 
problem, the strategies needed to solve it, the mathematical content/types of numbers used, 
and the sources from which data need to be obtained to solve the problem. 



(TC# 500.3ANASSM) 



Lester, Frank K. Jr., and Diana Lambdin-Kroll. Assessing Student Growth in Mathematical 
Problem Solving. Located in: Assessing Higher Order Thinking in Mathematics , 
Gerald Kulm (Ed.), 1990. Available from: American Association for the Advancement 
of Science, 1333 H Street NW, Washington, DC 20005, (301) 645-5643. 

The authors present a model of factors that influence problem-solving performance, and 
discuss several problem-solving assessment techniques. 

A good assessment program in math should collect information about the following: affect 
(attitudes, preferences, and beliefs), and cognitive/processes ability to get the right answer 
(both whether they get the right answer, and the strategies used). The program should also 
systematically define and cover the features of tasks (problem type, math content, required 
strategies, etc.) since these affect performance and should be reflected in instruction. 

In order to gather information on these three categories of factors, the authors briefly review: 
observations, interviews, student self-reports, and holistic and analytic scoring of 
performances. They recommend against multiple-choice questions. 

This paper is a general theoretical discussion; no actual tasks, problems or scoring guidelines 
are provided. 

(TC# 500.6ASSSTG) 



Marshall, Sandra P. Assessing Knowledge Structures in Mathematics: A Cognitive Science 
Perspective. Located in: Cognitive Assessment of Language and Mathematics 
Outcomes , Sue Legg & James Algina (Eds.), 1990. Available from: Ablex Publishing 
Company, 355 Chestnut St., Norwood, NJ 07648. 

This article discusses the implications of recent advances in cognitive science for mathematics 
assessment. The goal in using this research to develop assessment techniques is to determine 
the extent to which students have acquired specific cognitive skills rather than merely whether 
they can correctly solve particular problems. 

Cognitive theory holds that people solve problems by using three knowledge structures- 
declarative (facts), procedural (algorithms and production rules), and schema (frames that 
relate facts and production rules). To solve a problem, a person must first find the right 
schema, must then correctly implement a set of production rules, and must have stored 
correctly the facts and knowledge required to carry out the necessr algorithms specified by 
the production rules. Errors can occur in any of these three areas. 
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Researchers are currently engaged in specifying these knowledge structures in such detail that 
they can develop computer simulations that can, first, solve problems, and second, reproduce 
student errors by leaving out or altering various parts of the necessary structures. In this way, 
errors in student responses can be tracked back to the erroneous structure used. The author 
specifically mentions work in the area of simple arithmetic operations, geometry, and word 
problems. 

Additionally, the author discusses two other ways of assessing these things in students- 
reaction time (to assess how automatic a function is); and multiple-choice problems (e.g., 
"which of the following problems can be solved in the same way as the one stated above?" to 
get at schema knowledge). Some time is spent with multiple-choice problems to explore 
various types of problems and the technical issues that arise with them. 

It should be pointed out that all these procedures are experimental; none have progressed to 
the point where there is a final product that can be ordered and installed. 

(TC# 500.6ASSKNS) 

Marshall, Sandra P. The Assessment of Schema Knowledge for Arithmetic Story Problems: 
A Cognitive Science Perspective, 1990. Located in: Assessing Higher Order Thinking in 
Mathematics , Gerald Kulm (Ed.). Available from: American Association for the 
Advancement of Science, 1333 H Street NW, Washington, DC 20005, (301) 645-5643. 

The Story Problem Solver (SPS) was created to support instruction based on a theory of 
memory architecture called schemata. Under such theories, human memory consists of 
networks of related pieces of information. Each network is a schema-a collection of well- 
connected facts, features, algorithms, skills, and/or strategies. 

Adult students are explicitly taught five problem-solving schemas and how to recognize which 
schema is represented by a story problem. SPS is a computerized assessment method in which 
several different item types are used: students pick out the schema or general solution 
strategy that fits a given story problem, decide which information in the story problem fits into 
the various frames of the schema, identify the steps needed to solve a problem, and decide 
whether the necessary information is given in the problem. 

Some of the schema shells and item types are given as examples. No technical information is 
included. 

(TC# ^00.3ASSOFS) 
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Maryland Department of Education. Maryland School Performance Assessment Program, 
1992. Available from: Gail Lynn Goldberg, Maryland Department of Education, 
Maryland School Performance Assessment Program, 200 W. Baltimore St., Baltimore, 
MD 21201,(410)333-2000. 

Maryland has released six performance tasks that illustrate the 1992 assessment. This review 
is based on three of them, one task at each of grades 3, 5 and 8. The tasks are integrated 
across subject areas and use some combination of information and skills in science, math, 
writing, reading, and social studies. The three tasks we have relate to the weather (Grade 3), 
snowy regions of the country (Grade 5) and collisions (Grade 8). Each task has both 
individual and group work and proceeds through a series of exercises that require reading, 
designing and conducting experiments, observing and recording information, and writing up 
results. 

Student responses are scored using two basic approaches: generalized holistic or analytical 
trait scoring for the "big" outcomes such as communication skills, problem solving, science 
process skills, and reasoning; and specific holistic ratings of conceptual knowledge and 
applications. For example, the task on collisions is scored both for knowledge of the concepts 
of mass and rate/distance, and for general science process skills (collecting and organizing 
data, and observation) and communication skills. Thus, some scoring guides are generalized 
across tasks, and some list specific features from individual tasks to v/atch for. 

The materials we have allude to anchor performances and training materials, but these are not 
included in our samples. Neither information about student performance, nor technical 
information about the tests is included. 

(TC# 060.3MARSCP) 

Maryland State Department of Education. Scoring MSPAP (Maryland School Performance 
Assessment Program): A Teacher's Guide, 1993. Available from: Gail Lynn Goldberg, 
Maryland Department of Education, Maryland School Performance Assessment 
Program, 200 W. Baltimore St., Baltimore, MD 21201, (410) 333-2000. 

This document presents information about the 1993 MSPAP in grades 3, 5, and 8: 
philosophy, general approach, sample tasks, and performance criteria. There are sample tasks, 
performance criteria and student responses for the following areas: expository, persuasive 
and expressive writing, read' 3 comprehension, math, science, and social studies. 

Scoring can be done three different ways depending on the task: generalized scoring rubrics 
that can be used across tasks (e.g., persuasive writing); generalized scoring rules that are not 
as detailed as rubrics (e.g., language usage); and scoring keys that are task-specific (e.g., 
many math tasks are scored for the degree of "correctness" of the response) 

No technical information is included. 

(TC# 000.3SCOMST) 
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Massell, Diane. Setting Standards in Mathematics and Social Studies. Located in: 
Education and Urban Society 26, February 1994, pp. 118-140. 



This article describes, analyzes, and contrasts two efforts to set curriculum standards-the 
National Council of Teachers of Mathematics (NCTM) efforts in math, and the state of 
California's efforts in history-social studies. It describes the history of development and 
provides ideas on what it takes to have a successful development effort. 

(TC# 000.5SETSTM) 

Mathematical Science Education Board, National Research Council. Measuring Up- 

Prototypes for Mathematics Assessment, 1993. Available from: National Academy Press, 
2101 Constitution Ave, NW, Washington, DC 20418, (800) 624-6242. 

Measuring Up is designed to illustrate performance assessment tasks that could be used with 
fourth graders to support reform efforts in mathematics instruction. The book presents 13 
prototype assessments which are meant to be examples of assessment possibilities, not 
examples of ready-to-use assessments that provide an adequate sample of the NCTM 
standards. 

Each task description contains the following information: time allotment, student grouping, 
assumed background knowledge of students, task description, student materials, 
rationale/explanation, and "protorubric." The 13 assessments have the following features. 
( 1 ) they take 1-3 class periods; (2) some require collaboration; (3) most require student 
explanations; (4) they present a series of questions related to a general activity; (5) most have 
a single correct or best response, although a few are more open-ended; and (6) some, have 
Spanish translations. 

Scoring is based on "protorubrics," beginning scoring guides that are not yet fully fleshed out. 
All are task-specific and use a three-point scale-high, medium, or low response. Abstracting 
across "protorubrics," the following types of things are included in the "high" category: 
correctness of response, evidence of conceptual understanding, flexibility of thinking, clarity 
of p-esenting results, and problem solving. (However, the "protorubrics" do not themselves 
identify their own content in this manner, nor do all include everything on this list.) The 
"protorubrics" are not, in themselves sufficient to interpret results; the "rationale" section for 
each assessment is also necessary. 

Tasks were pilot-tested to ensure that they work as intended. No other technical information 
is available. Many samples of student responses are included. 

(TO 500.3MEAUPP) 
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McTighe, Jay. Maryland Assessment Consortium: A Collaborative Approach to 

Performance Assessment, 1991. Available from: Maryland Assessment Consortium, 
c/o Frederick County Public Schools, 115 E. Church St., Frederick, MD 21701, 
(301) 694-1337. 

This entry contains handouts from a presentation by the author in 1991 . The following topics 
are covered. 

1 . A description of the consortium—what it is and what it does. 

2. An overview of the process used for developing performance tasks, and review criteria 
for performance tasks. 

3. Examples of three performance assessment tasks developed by the consortium: one 
math problem-solving task for grade six and two fifth grade reading tasks. All tasks are 
scored using a four-point holistic scoring guide. Scoring appears to be generalized 
rather than tied to individual tasks. The reading tasks, for example, are scored using the 
same, generalized scoring guide. 

(TC# 500.3MARASC) 

McTighe, Jay. Teaching and Testing in Maryland Today: Education for the 21st Century, 
1992. Available from: Maryland Assessment Consortium, c/o Frederick County Public 
Schools, 115 E. Church St., Frederick, MD 21701, (301) 694-1337. 

This 13-minute video is designed to introduce parents and community members to 
performance assessment. 

(TC# 150.6TEATEMv) A2. A4 

Mead, Nancy. IAEP (International Assessment of Educational Progress) Performance 
Assessment (Science and Math), 1992. Available from: Educational Testing Service, 
Rosedale Rd., Princeton, NJ 08541, (609) 734-1526. 

This document supplements the report by Brian Semple (also described in this bibliography) 
(TC# 600.6PERASS). The document contains the administrators manual, scoring guide, 
equipment cards, and released items from the Second International Assessment of Educational 
Progress in science and mathematics. 

(TC# 500.3IAEPPA) 
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Med rich, Elliott A., and Jeanne E. Griffith. International Mathematics and Science 
Assessments: What Have We Learned?, 1992. Available from: National Technical 
Information Service, US Department of Commerce, 5285 Port Royal Rd., Springfield, 
VA 22 1 6 1 , (703) 487-4650. 

This report provides a description of the international assessments of math and science (First 
International Mathematics and Science Studies, 1960's; Second International Mathematics and 
Science Studies, 1980's; and First International Assessment of Educational Progress, 1988), 
some of their findings, and issues surrounding the collection and analysis of these data. It also 
offers suggestions about ways in which new data collection procedures could improve the 
quality of the surveys and the utility of future reports. 

(TC# 000.6INTMAS) 

Meltzer, L. J. Surveys of Problem-Solving & Educational Skills, 1987. Available from: 
Educator's Publishing Service, Inc., 75 Moulton St., Cambridge, MA 02138. 

Although this is an individual test published primarily for diagnosing learning disabilities for 
students aged 9-14, it has some interesting ideas that could be more generally applied. There 
are two parts to the test— a more-or-less standard individualized aptitude test, and a series of 
achievement subtests. The math subtest involves a fairly standard test of computation. The 
interesting part comes in the scoring. Each problem is scored on choice of correct operations, 
ability to complete the word problem, efficiency of mental computation, self-monitoring, self- 
correction, attention to operational signs, and attention to detail (one point for evidence of 
each trait). 

After the entire subtest is administered, the teacher is guided through analysis of the student's 
strategies in completing the task—efficiency of approaching tasks, flexibility in applying 
strategies, style of approaching tasks, attention to the task, and responsiveness during 
assessment. (Each area is assigned a maximum of three points for the presence or absence of 
three specific features of performance. For example, under "efficiency" the students get a 
point if he or she does not need frequent repeating of instructions, a second point if the 
student implements the directions rapidly, and a third point if the student perseveres to 
complete the task.) Examples of scoring are included. 

A fair amount of technical information is included. This covers typical performance, factor 
analysis, inter-rater reliability, relationship to other measures of performance, and comparison 
of clinical groups 

(T0 010.3SUROFP) 
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Miller, Edward, Ed. The Push for New Standards Provokes Hope and Fear— and Both Are 
Justified Located in: The Harvard Education Letter 9, September/October 1993, 
pp. 1-6. 

The author's position is that new standards, in and of themselves, will not reform or improve 
American education in math. We also need to pay equal attention to developing teachers' 
skills and to providing all students with equal opportunity to learn. Roadblocks to real reform 
include: (1) outmoded tests; (2) lack of teacher sophistication in math; (3) pressure to cover 
all the material in textbooks; (4) lack of time for planning and learning; and (5) lack of 
materials and resources. The article also includes other topics such as the need for standards 
that emphasize different skills and not just attaining higher levels of traditional skills, lessons 
from other countries, and vignettes demonstrating the various points made in the article. 

(TC# 500.5PUSNES) 

Models of Authentic Assessment Working Group. Superitem Tests in Mathematics, 
undated. Available from: National Center for Research in Mathematical Sciences 
Education, 1025 W. Johnson St., Madison, WI 53706, (608) 263-3605 

The document we received is a set of 20 open-response tasks that are designed to measure the 
statistics, measurement and geometry knowledge of middle school students (grades 7-9). 
Each of the 20 tasks has a set of four questions, each of which has only a single correct 
answer. From a sample problem, it appears that all responses are scored right/wrong using 
task-specific scoring guides. 

The materials we received did not contain a description of which specific goals were covered 
by each question, sample student responses, scoring guides, contextual information, nor 
technical information. 

(TC# 500.3SUPTM) 

Mullen, Kenneth B. Free-Response Mathematics Test, 1992. Available fnm: American 
College Testing Program, PO Box 168, Iowa City, I A 52240, (319) 337-1051. 

This was a paper presented at the annual meeting of the National Cou icil on Measurement in 
Education, San Francisco, April 1992. 

This paper reports on a study by ACT that compares multiple-choice, open-response, and 
gridded response item formats on reliability, difficulty and discrimination. In gridded response 
items, students fill in "bubbles" that correspond to the answer rather than choosing the answer 
from a given list. "Testlets" were designed to cover the same content and have the same test 
length for each format. Results indicated that all formats had about the same reliability; there 
was good rater agreement on the open-ended problems; and grid and open-ended problems 
discriminated better between students with different achievement levels. The correlation 
between performances on the various types of items ranged ffom 0.5 to 0.7 
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A few sample problems are provided. All open-response questions used scoring criteria that 
emphasize degree of correctness of the response and were tied to the task (i.e., there was a 
different scoring guide for each problem). 

(TC# 500.3FREREM) 

Mumme, Judy. Portfolio Assessment in Mathematics, 1990. Available from: California 
Mathematics Project, University of California—Santa Barbara, 522 University Rd., 
Santa Barbara, CA 93106, (805) 961-3190. 

This booklet describes what mathematical portfolios are, what might go into such portfolios, 
how items should be selected, the role of student self-reflection, and what might be looked for 
in a portfolio. Many student samples are provided. Criteria for evaluating portfolios include, 
evidence of mathematical thinking, quality of activities and investigation, and variety of 
approaches and investigations. No technical information is included. 

(TC# 500.6PORASI) 

National Science Foundation. Educating Americans for the 21st Century: A Plan of Action 
for Improving Mathematics, Science and Technology Education, 1983. Available from: 
National Science Board Commission on Precollege Education in Mathematics, Science 
and Technology, Forms & Publications Unit, 1800 G St. NW, Room 232, 
Washington, DC 20550, (202) 357-3619. 

This is not strictly a document regarding assessment, but rather a statement of what students 
need to know and be able to do in science and math. As such, it also provides an outline for 
what assessments should measure. 

(TC# 000.5EDUAMF) 



Nicoll, Richard, Kathy Jacobson, Jim Rybolt. Curriculum-Based Alternative Assessment of 
Mathematics (CRAAM) {Third Grade and Sixth Grade/ A Report to Teachers, April 1993. 
Available from: Mt. Diablo Unified School District, 1936 Carlotta Dr., Concord, CA 
94519, (510) 682-8000, ext. 4135. 

This review is based on reports of the 1993 third and sixth grade assessments. Students 
respond to three extended problems, some of which have right/wrong answers and some of 
which are more generative. For example, one third grade problem requires students to plan a 
city. Students are directed to list "buildings and places needed to have a working 
community," pick 8-10 from the list, place these on a map, and describe the rationale for the 
placement. Students do both group and individual work. 

A generalized, holistic, six-point scoring guide is tailored to individual problems. In the 
generalized version, a "6" is: "complete response with a clear, coherent, unambiguous, and 
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elegant explanation; includes a clear and simplified diagram when appropriate; communicates 
effectively to the reader; shows understanding of the open-ended problem's mathematical 
ideas and processes; identifies all the important elements of the problem." The document 
includes three scored student responses for each problem. 

The document also includes a rationale for alternative assessment, and the context for the Mt. 
Diablo assessment. No technical information is included. The author has given educators 
permission to copy this document for their own use. 

(TC# 500.3MTDIAC2) 

Nicholls, John G., Paul Cobb, Erna Yackel, et al. Students' Theories About Mathematics 
and Their Mathematical Knowledge: Multiple Dimensions of Assessment. Located in: 
Assessing Higher Order Thinking in Mathematics , Gerald Kulm (Ed.), 1990. Available 
from: American Association for the Advancement of Science, 1333 H St. NW, 
Washington, DC 20005, (301) 645-5643. 

This paper reports on a series of studies on student attitudes toward mathematics and their 
relationship to mathematical knowledge and understanding. Dimensions of attitudes toward 
math were: 

1 . How motivated students are to do math 

2. Student beliefs about what causes success in math 

3. Student views of the benefits of learning math. 
All items are included. 

(TC# 500.3STUTHA) 



Oregon Department of Education. Oregon Dimensions of Problem Solving, 1994. Available 
from: Office of Assessment and Technology, Oregon Department of Education, 
255 Capitol St NE, Salem, OR 97310, (503) 378-8004. 

The Oregon Department of Education began giving open-ended math problems to a sample of 
students in grades 3, 5, 8, and 1 1 in 1992. The five short, written problems used in each 
grade in 1992 are included in this document, as are student instructions. Responses are 
scored on four dimensions, or traits: (1 ) conceptual understanding of the problem— the ability 
to interpret the problem and select appropriate information to apply a strategy for solution, 
(2) procedural knowledge-the ability to demonstrate appropriate use of math; (3) skills to 
solve the problem; and (4) communication-the ability to use math symbols well and the ability 
to explain the problem solution. 
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Each trait is scored on a scale of 1-5. The 1994 scoring guides are included in this document 
along with one sample student problem. No technical information nor anchor papers are 
included. 

(TC# 500.3ORDIPS) 

Pandey, Tej. Power Items and the Alignment of Curriculum and Assessment. Located in: 
Assessing Higher Order Thinking in Mathematics , Gerald Kulm (Ed.), 1990. Available 
from: American Association for the Advancement of Science, 1333 H St. NW, 
Washington, DC 20005, (301) 645-5643. 

The author presents a philosophy and approach for thinking about the development of a test . 
of mathematics problem solving, and provides some examples of multiple-choice and short- 
answer "power" questions developed by the California Assessment Program. 

The author maintains that typical content-by-process matrices used to specify the content of 
tests tend to result in tests that measure minuscule pieces of information that are fragmented 
and non-integrated. The author prefers to have assessment tasks that are broader in focus and 
cut across several process/content areas, so that in order to get the right answer, students 
must use skills like organizing information, representing problems, and using strategies. 

(TC# 500.6POWITA) 

Pandey, Tej. A Sampler of Mathematics Assessment, 1991. Available from: California 
Department of Education, Bureau of Publications, Sales Unit, PO Box 944272, 
Sacramento, CA 94244, (916) 445-1260. 

This sampler describes the types of assessment that the California Assessment Program (CAP) 
is proposing to support curricular reforms. Illustrated and discussed are open-ended 
problems, enhanced multiple-choice questions, investigations, and portfolios. These four 
types of activities are intended to measure mathematical understandings that students develop 
over a period of several years. 

This monograph includes a definition of "mathematical power"-the ultimate goal of 
mathematics instruction, guidance in the characteristics of assessment tasks that will 
encourage and measure power, a few sample student responses to problems, and help with 
implementation of alternative assessment. 

All performance-based techniques will use a six-point holistic scale. This scale is briefly 
described. The scale will be tailored for individual tasks. 

(TC# 500.3SAMMAA) 
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Paulson, Leon. Portfolio Guidelines in Primary Math, 1992. Available from: Multnomah 
County Educational Service District, PO Box 301039, Portland, OR 97220, 
(503) 255-1842. 



This monograph provides some assistance with getting started with portfolios in the primary 
grades. The author believes that the most important purpose for mathematics portfolios is to 
prompt students to take control of their own learning. Therefore, the student should be in 
control of the portfolio. (The author, however, also points out that there might be other 
audiences and purposes for the portfolios that might have to be addressed.) 

The author provides some ideas for tasks that students could do to generate material for the 
portfolio, provides some very practical suggestions for getting started, gives ideas for 
activities to encourage student self-reflection, and shows some draft holistic criteria for 
evaluating portfolios. 

An example of the user-friendly way this monograph provides practical help is: "Remember, 
the portfolio is telling a story. Each item in a portfolio is there for a reason. It should not 
require a mind reader to figure out why it is there. A portfolio entry includes a piece of work 
plus information that makes its significance clear— the reason it was selected, the learning 
goals illustrated, student self-reflections, and (always!) the date." 

Note: A new version is due out in the fall of 1994. 

(TC# 500.6PORGUP) 



Paulson, Leon, and Pearl Paulson. An Afternoon to Remember: A Portfolio Open House for 
Emotionally Disabled Students, 1992. Available from: Multnomah County Educational 
Service District, PO Box 301039, Portland, OR 97220, (503) 255-1842. 

Reynolds School District adapted Crow Island's "portfolio night" for use with severely 
emotionally disabled students. This paper describes how the afternoon was set up, what 
happened, student debriefing sessions, and changes in format based on student comments. 

(TC# 000.6AFTREP) 



Pfeiffer, Sherron. NIM Game Project, 1992. Available from: Southeast EQUALS, 
14 Thornapple Dr., Hendersonville, NC 28739, (704) 692-4078. 

The assessments described in this document are appropriate for upper elementary and middle 
school students. Two project tasks are included, one individual and one group. The projects 
require students to create a game that requires application of math skills. These extended 
projects are used after students have had many opportunities to work with different kinds of 
NIM games. The extended nature of the project emphasizes persistence and the importance 
of quality products. Projects become part of a portfolio that shows growth over time. 
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The projects are scored using criteria specific to these tasks. The criteria revolve around the 
quality of the game and its usefulness in teaching the math skills specified. The project 
instructions and scoring guide are included. No sample student work nor technical 
information is included. This exercise is part of a book of teaching strategies produced by and 
available from the author: Successful Teaching Strategies. 

The author has given educators permission to copy this document for their own use. 
(TC# 500.3NJMGAP) 

Pritchard, Diane. Student Portfolios- Are They Worth the Trouble?, 1992. Available from: 
Sisters Middle School, PO Box 555, Sisters, OR 97759, (503) 549-8521. 

This paper was written by a middle school math and English teacher. It provides practical 
help with how to set up a portfolio system in math by describing her purpose for having a 
portfolio, the types of activities included, and activities to get students to self-reflect 
(including an idea for tests). 

(TC# 500.3STUPOT) 

Psychological Corporation, The. GOALS: A Performance-Based Measure of Achievement, 
1992. Available from: The Psychological Corporation, Order Service Center, 
PO Box 839954, San Antonio, TX 78283, (800) 228-0752. 

GOALS is a series of open-response questions that can be used alone or in conjunction with 
the MAT-7 or SAT-8, or any achievement test. Three forms are available for 1 1 levels of the 
test covering grades 1-12 in the subject areas of science, math, social studies, language and 
reading. Each test (except language) has ten items. The manual states that the math 
questions assess student problem solving, communication, reasoning, connections to other 
subjects, estimation, numeration, geometry, patterns, statistics, probability and algebra. Tasks 
are multiple, short problems. The manual draws the distinction between the approach taken in 
GOALS (efficiency in large-scale assessment), and the related publication "Integrated 
Assessment System" which has fewer tasks pursued in more depth. 

Responses are scored on a scale of 0-3, where 0 is "response is incorrect" and 3 is "accurate 
and complete with supporting information." The scoring guide is generalized and is used for 
all problems. Scoring can be done locally or by the publisher. There is good assistance with 
scoring philosophy and procedures. There are two sample student performances for each 
score point for each question. 

The holistic scales are combined in various ways to provide indicators of overall conceptual 
understanding and various specific aspects of problem solving and using procedures. These 
are, however, not scored directly. Rather, it is analogous to multiple-choice tests in which the 
correct items are combined in various ways to give subtest scores. 
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Both norm-referenced (percentiles) and criterion-referenced (how students perform on 
specific concepts) score reports are available. A full line of report types (individual, summary, 
etc.) are available. 

The materials we obtained did not furnish any technical information about the test itself. 
(TC# 510.3GOALS) 



Psychological Corporation, The. Integrated Assessment System: Mathematical Performance 
Assessment, 1991. Available from: The Psychological Corporation, Order Service 
Center, PO Box 839954, San Antonio, TX 78283, (800) 228-0752. 

This is a series of 14 tasks designed to be used with students in grades 2-8. Two task 
booklets were designed for each grade level, but can also be used in spring testing of the 
grade below or fall testing of the grade above. Each task booklet presents a problem situation 
that is expanded on and applied to a series of questions. For example, various task booklets 
focus on symmetry, breaking a tie in an election, planning an orchard to maximize yield, and 
bar codes. Questions involve such things as figuring out an answer and explaining how the 
solution was reached, and generating a principle and applying it to a new situation. 

Solutions are scored either holistically (0-6) or analytically (four, 4-point scales). The 
performance criteria represent generalized features of problem solving and so can be used to 
score performance on any task. The holistic scale is used to provide an overall picture of 
performance; raters look for quality of work, evidence of understanding' of concepts, logical 
reasoning, and correct computations. The analytical traits are: reasoning, conceptual 
knowledge, communication, and procedures. Scoring can be done either locally or by the 
publisher. 

The set of materials we obtained includes a brief description of the scoring rubrics and one 
example of a scored student test. Technical information was not included. 

(TC# 500.3INTASM) 



Riverside Publishing Company, The. Arizona Student Assessment Plan— Mathematics. 
Available from: The Riverside Publishing Co., 8420 Bryn Mawr Ave., Chicago, IL 
60631, (800) 323-9540. 

Arizona is developing open-response, on-demand assessments in math, reading, writing, social 
studies, and science for grades 3, 8, and 12. The math test has been used for about two years; 
the science test is still under development. The mathematics test requires no manipulatives or 
equipment. Students provide short answers to a series of questions surrounding a common 
theme, such as a rock climbing competition or a "pizza spinner." They also sometimes 
provide explanations for responses or show computations. 
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Scoring is task-specific. Points are assigned for the correctness of the response or the number 
of responses given. 

The tests were developed by Riverside and are under copyright restriction until at least 1996. 
Permission to use any part of the assessment must be granted both by Riverside and the 
Arizona Department of Education. 

(TOt 500.3ARISTM) -- IN-HOUSE ONLY 

Riverside Publishing Company, The. California Program for Learning Assessment- 
Mathematics Performance Assessments, 1994. Available from: The Riverside 
Publishing Company, 8420 Bryn Mawr Ave., Chicago, IL 60631, (800) 323-9540. 

The California Program for Learning Assessment addresses language arts and mathematics in 
eight levels for grades 3-10. The math tests are designed to be given in 45 minutes and have 
two parts. Part 1 contains two open-ended math problems which take approximately 30 
minutes to complete. Part 2 has seven multiple-step, multiple-choice problems which take 
approximately 15 minutes to complete. The test is designed to assess problem solving, 
application of knowledge, and communication skills rather than knowledge of specific facts or 
operations. The open-ended problems require written responses and are completed 
individually. Both open-ended and multiple-choice questions are thought-provoking. 

Open-ended responses are scored using a 0-4 point, generalized, holistic rubric where "4" 
"Shows a complete understanding of the problem and addresses all relevant mathematical 
ideas. Exhibits sound reasoning and draws logical conclusions. Communicates clearly 
through the use of appropriate charts, graphs, diagrams, illustrations, and/or words. Provides 
computation (where required) adequate for the solution of the problem." Although somewhat 
sketchy, this rubric attempts to address the "big" outcomes in the NCTM standards. To help 
the scorer, the general rubric is tailored to each particular problem. 

The materials we received mention a pilot test in 17 California schools, but no details are 

given. 

(TC# 500.3CALPRL) IN-HOUSE USE ONLY 

Riverside Publishing Company, The. Performance assessments for ITBS, TAP and ITED 
[various levels and subject areas/, 1993. Available from: The Riverside Publishing 
Company, 8420 Bryn Mawr Ave., Chicago, IL 60631, (800) 323-9540. 

Riverside is publishing a series of open-response items in the areas of social studies, science, 
mathematics, and language arts. Nine levels are available for grades 1-12. They supplement 
achievement test batteries available from the publisher: ITBS, TAP, and ITED. Each level 
uses a scenario to generate a series of related questions, some of which have only one right 
answer, and others of which are more open-ended and generative. Tests take 1 V 2 to 2 hours 
depending on grade level 
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No information about scoring, no sample student performances, and no technical information 
were included in the materials we received. However, the publisher's catalog indicates that 
scoring materials are available and that the tests are normed. 

(TC# 060.3PERAST) IN-HOUSE USE ONLY 

Romberg, Thomas A. Assessing Mathematics Competence and Achievement, 1989. 

Available from: National Center for Research in Mathematical Sciences Education, 
Wisconsin Center for Educational Research, University of Wisconsin, School of 
Education, 1025 W. Johnson St., Madison, WI 53706, (608) 263-4200. 

This paper describes the author's view of what it means to be literate mathematically. It then 
describes the instructional and assessment implications of this goal. The author believes that 
we need to assess not only mathematical knowledge but also the structure of the knowledge. 

(TC# 500.5ASSMAC) 



Romberg, Thomas A. The Domain Knowledge Strategy for Mathematical Assessment, 1987. 
Available from: National Center for Research in Mathematical Sciences Education, 
Wisconsin Center for Educational Research, School of Education, 1025 W. Johnson St., 
Madison, WI 53706, (608) 263-4200. 

This document provides a brief overview of the "Domain Knowledge" strategy used by the 
National Center for Research in Mathematical Sciences Education to assess math knowledge 
of students. This approach is contrasted to the typically used "Content by Behavior Matrix" 
approach in which content topics are crossed with behavior (usually some form of Bloom's 
taxonomy). The author maintains that this approach is outdated; the behavior dimension fails 
to reflect contemporary notions of how information is processed and the content dimension is 
an inadequate way to describe what is meant by "knowing mathematics." 

The "Domain Knowledge" approach involves making a "map" or network of a concept 
domain. This reflects a more integrated and coherent picture about knowledge. These maps 
can be used to generate tasks, assessment criteria, and formats that get at both "correctness" 
of responses and the strategies used to arrive at the answer. 

(TC# 500.6DOMKNS) 
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Romberg, Thomas A. Evaluation: A Coat of Many Colors, 1988. Available from: National 
Center for Research in Mathematical Sciences Education, Wisconsin Center for 
Educational Research, University of Wisconsin, School of Education, 1025 W. Johnson 
St., Madison, WI 53706, (608) 263-4200. Also located in: Mathematics Assessment and 
Evaluation: Imperatives for Mathematics Educators , Thomas A. Romberg (Ed.), 1992. 
Available from: State University of New York Press, State University Plaza, Albany, 
NY 12246. 

This paper describes the impact of assessment information on decision making and describes 
the ways in which assessment must change if it is to have a positive impact on such decisions. 

(TCM 500.6EVACOM) 

Romberg, Thomas A. Mathematics Assessment and Evaluation: Imperatives for 

Mathematics Educators, 1992. Available from: State University of New York Press, 
State University Plaza, Albany, NY 12246. 

This book covers several interesting topics with respect to assessment in math. Specifically: 

1 How tests communicate what is valued 

2 How current tests will not promote the recommendations in the NCTM standards 

3 Various considerations when developing tests: calculators, how to adequately model 
knowledgeable students, etc 

4 Setting up assessment that is intended to influence instruction 

Although authoritative, this book is written in a very academic style, which makes it less 
accessible to general readers. Articles that are most relevant to this bibliography are entered 
separately. 

(TO 500.6MATASE) 

Romberg, Thomas A., and Linda D. Wilson. Alignment of Tests with the Standards. 
Located in: Arithmetic Teacher , September 1992, pp. 18-22. 

The authors make the argument that teachers teach to tests. Therefore, if we want the NCTM 
standards to be implemented we need to have tests that reflect the standards. The authors 
feel that many current norm-referenced tests do not match the standards. Finally, they present 
tarks from several innovative assessments that they feel do reflect the standards. 

(TC# 500.6ALITEW) 
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Romberg, Thomas A., Linda D. Wilson, 'Mamphono Khaketla, et al. Curriculum and Test 
Alignment. Located in: Mathematics Assessment and Evaluation: Imperatives for 
Mathematics Educators , Thomas A. Romberg (Ed.), 1992. Available from: State 
University of New York Press, State University Plaza, Albany, NY 12246. 

This article reports on two studies on the alignment of current standardized tests and 
alternative assessments to the NCTM standards. Results showed that current standardized 
tests are weak in five of six content and process areas, and place too much emphasis on 
procedures and not enough on concepts. The authors present several examples of test 
questions that they feel do match the standards. 

(TC# 500.6CURTEA) 



Romberg, Thomas A., E. Anne Zarinnia, and Steven R. Williams. The Influence of 
Mandated Testing on Mathematics Instruction: Grade 8 Teachers' Perceptions, 1989. 
Available from: National Center for Research in Mathematical Sciences Education, 
Wisconsin Center for Educational Research, School of Education, 1025 W. Johnson St., 
Madison, WI 53706, (608) 263-4200. 

This monograph reports on the first of a sequence of studies on mandated testing in 
mathematics. This study was a large-scale questionnaire survey to find out from Grade 8 
teachers how influential mandated testing was on their teaching of mathematics. The results 
of the study showed that nearly 70 percent of the teachers reported that their students take a 
mandated test. Secondly, because teachers know the form and character of the tests their 
students take, most teachers make changes in their teaching to reflect this knowledge. Third, 
the kinds of changes teachers make are in contrast to the recommendations made by the 
NCTM standards. Specific examples are given. 

Although this paper does not describe an alternative assessment device, it does provide 
reasons for seeking alternative ways of assessing math. 

(TC# 500.6INFMAT) 



Schoenfeld, Alan H. Teaching Mathematical Thinking and Problem Solving. Located in: 
Toward the Thinking Curriculum: Current Cognitive Research , Loren B. Resnick & 
Leopold E. Klopfer (Eds.), 1989. Available from: Association for Supervision and 
Curriculum Development, 1250 N. Pitt St., Alexandria, VA 22314-1403, (703) 549-9110. 

Although this article is more about defining what mathematical problem solving is than about 
assessment, it presents an interesting visual way to represent how students spend their time 
when solving a problem. It also compares a plot of time use for a good problem solver to a 
plot for an inefficient problem solver. 

Essentially, the plotting procedure involves tracking the sequence in which people use 
different steps in the problem-solving process (reading the problem, analyzing the problem, 



ERIC 



NWREL, August 1994 
Test Center, (503) 275-9582 



52 



Mathematics 



exploring a solution strategy, planning, implementing a strategy, and verifying the results) and 
the amount of time spent on each. Good problem solvers spend a lot of time analyzing and 
planning, with many self-checks on "how it is going." Poor problem solvers tend to fixate on 
a possible line of attack and pursue it relentlessly even when it is clearly not going well. 
Additionally, there are very few stops to self-check on how it is going 

(TC# 500.5STOWTET) 

Scottish Examination Board. Standard Grade - Revised Arrangements in Mathematics, 
1987. Available from: Mr. K. Hutchon, Examination Officer, Ironmills Rd., Dalkeith, 
Midlothian, Edinburgh, Scotland, EH22 1LE, UK (031) 663-6601. 

The Scottish Examination Board prepares end-of-course tests for a variety of high school 
subjects to certify level of student competence. The course syllabus for mathematics calls for 
coverage of number, money, measurement, geometry, trigonometry, algebra, and 
graphs/tables. The goals of the course are: knowledge/understanding, reasoning/applications, 
and investigating. There are two main parts of the assessment in math— written tests 
(developed by the Examination Board) and performance assessments (conducted by teachers 
according to specifications developed by the Examination Board). The two parts are 
combined to rate student competence on a scale of 1-7 (1 being high), both overall and for 
each goal. 

On-demand written tests, developed each year, cover knowledge/understanding and 
reasoning/applications. Three levels of the test are available: Foundation, General and Credit. 
Depending on the percent correct score, students can obtain ratings of 1 or 2 on the Credit 
level, 3 or 4 on the General level, and 5 or 6 on the Foundation level. All questions are short 
answer or multiple-choice and are scored for degree of correctness of the answer. 

The hands-on performance assessments must cover a range of activities including: the 
identification and use of real data; the use of measuring or drawing instruments; the 
recognition or exploration of a pattern, conjecture, or proof; and the formulation of a 
mathematical model. Candidates write up their investigations in the form of a report. 
Performances are scored on "understanding and organizing the task," "carrying out the task," 
and "communication " A total of 1 2 marks (points) is available. General criteria for level 
designation are provided as well as the point conversions described above. 

The package of materials we received included the course syllabus, specifications for the 
written and performance assessments, and copies of the written tests for 1993. It did not 
include technical information nor sample student responses. 

(TC# 500.3MATSTG) 



NWKKL, August \ W4 
Test Center, (503) 275-95X2 



53 



57 



Mathematics 



Semple, Brian. Assessment of Achievement Programme— Mathematics: Second Survey 1988. 
Available from: Brian Semple, Principal Research Officer, New St. Andrews House, 
Room 4/51a, Edinburgh, Scotland EH1 3SY, UK (031) 244-4388. 

The "Assessment of Achievement Programme (AAP)" was established by the Scottish Office 
Education Department in 1981 to monitor the performance of pupils in grades 4, 7, and 9. 
The 1989 report "Mathematics: Second Survey 1988" reports on the 1988 mathematics 
assessment. The assessment covered geometry, algebra, estimation, and statistics (depending 
on grade level). 

Assessment tasks used two formats: written and practical. However, the report we have does 
not describe these two formats, nor explain how responses were scored. Schools in the 
assessment sample were also invited to comment on their mathematics program. 

The document we have includes the rationale for the assessment and description of student 
performance. No technical information is included. 

(TC# 500.3 ASSACP) 

Semple, Brian McLean. Performance Assessment: An International Experiment, 1991. 
Available from: Educational Testing Service, The Scottish Office, Education 
Department, Rosedale Rd., Princeton, NJ 08541, (609) 734-5686. 

Eight math and eight science tasks were given to a sample of thirteen-year-olds in five 
volunteer countries (Canada, England, Scotland, USSR, and Taiwan). This sample was 
drawn from the larger group involved in the main assessment. The purpose of the assessment 
was to provide an information base to participating countries to use as they saw fit, and to 
examine the use of performance assessments in the context of international studies. 

The 16 hands-on tasks are arranged in two 8-station circuits. Students spend about five 
minutes at each station performing a short task. Most tasks are "atomistic" in nature; they 
measure one small skill. For example, the 8 math tasks concentrate on measuring length, 
angles, and area, laying out a template on a piece of paper to maximize the number of shapes 
obtained, producing given figures from triangular cut-outs, etc. Some tasks require students 
to provide an explanation of what they did. All 16 tasks are included in this document, 
although some instructions are abbreviated and some diagrams are reduced in size, the 
complete tasks, administration and scoring guides are available from ETS. 

Most scoring is right/wrong; student explanations are summarized by descriptive categories. 
There is also observation of the products of students' work. 

Student summary statistics on each task are included. There is a brief summary of teacher 
reactions, student reactions, the relationship between student performance on various tasks, 
and the relationship between performance on the multiple-choice and performance portions of 
the test. A few sample student performances are included 
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(For related information, see Nancy Mead, also listed in this bibliography.) 



(TC# 600.3PERASS) 

Shoecraft, Paul. MOVE IT Math Concepts 1, 1988-1992. Available from: Mathematics 
Education Initiative, University of Houston- Victoria, 2506 E. Red River, Victoria, 
TX 77901,(512)576-3151. 

The MOVE IT Math Concepts 1 and MOVE IT Math Levels of Use Questionnaire together 
form a two-pronged assessment strategy to promote the implementation of Level 1 of the 
MOVE IT math program. MOVE IT math is a K-6, university-supported professional 
development program that advocates mathematics instruction based on the use of 
manipulates to address a wide variety of learning styles (e.g., visual, auditory, kinesthetic). 
It consists of three half-hour inservices (Level 1 is one of the three). 

The Moth Concepts I assessment instrument is a constructed response, paper-and-pencil test 
of the concepts of ( 1 ) equals as balanced, (2) exchanging "up" in base ten, (3) being an 
exchanging expert, and (4) exchanging "down" in base ten. A measurement item involving 
adding feet and inches is included to assess ability to take into account the context in which 
numbers appear. 

Copies of the Concepts 1 test, answers, and test results from six years of use are available. 
They may be used as it or modified, with appropriate citation to the authors. 

(TC# 500.3MOVITM) 

Silver, Edward A., and Jeremy Kilpatrick. Testing Mathematical Problem Solving. 
Located in: The Teaching and Assessing of Mathematical Problem Solving , Randall 
Charles and Edward Silver (Eds.), 1988. Available from: National Council of Teachers 
of Mathematics, Inc., 1906 Association Dr., Reston, VA 22091. 

This paper discusses two topics: how assessment can inform instructional decision making 
and how it communicates what we value. The authors propose that the National Assessment 
of Educational Progress and many other math tests do not provide the type of information 
needed for the improvement of mathematics instruction. The information useful for 
improvement of instruction would be types of errors kids make, how automatic mathematical 
processes are, and the cognitive structures and abilities associated with expertise in the 
domain being tested. 

(TC# 500.6TESMAP) 
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Stecher, Brian M. Describing Secondary Curriculum in Mathematics and Science: Current 
Status and Future Indicators, 1992. Available from: RAND, 1700 Main St., 
PO Box 2138, Santa Monica, CA 90407. 

The author describes what could go into an indicator system of the health of science and 
mathematics education. He concludes that current data sources for these indicators are 
inadequate. 

(TC# 000.6DESSEC) 

Stenmark, Jean Kerr. Mathematics Assessment: Myths, Models, Good Questions, and 
Practical Suggestions, 1991. Available from: National Council of Teachers of 
Mathematics, 1906 Association Dr., Reston, VA 22091. 

This monograph was designed for teachers in the elementary grades. It is a collection of 
examples of assessment techniques that focus on student thinking. Topics include the 
rationale for new ways of assessing mathematics, the necessity of integrating assessment and 
instruction, designing performance assessments (most emphasis is on designing the task, 
although sample holistic and analytical trait scoring systems are shown), what to look for 
during classroom observations and interactions (including questions to ask to get at various 
types of thinking), portfolios (including types of items to include and the types of information 
they can demonstrate about students, and criteria for evaluation), student self-assessment, and 
hints to make assessment work in the classroom. No technical information is provided. 

(TC# 500.3MATASM) 

Stuart Foundations Project Teachers. Language Arts and Science Performance Assessment 
Sampler, 1993. Available from: San Diego City Schools, 4100 Normal St., Room 3133, 
San Diego, CA 92103, (619) 298-8120. 

This document includes 12 performance assessment project tasks developed by teachers for 
students in grades K-3 and 4-8. Some of the tasks are interdisciplinary (science, math, 
writing,) and some are subject specific (mostly writing). The tasks themselves are interesting 
and may provide ideas for others developing performance assessments. This document does 
not include rationale or context for the tasks, technical information, or sample student work. 
The performance criteria listed for each project are sketchy. 

(TC# 100.3LANARS) 
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Surber, John R. Mapping as a Testing and Diagnostic Device, 1984. Located in: Spatial 
Learning Strategies-Techniques. Applications, and Related Issues , C. D. Holley & D. F. 
Dansereau (Eds.). Available from: Academic Press, 1250 6th Ave., San Diego, 
CA 92101. 

The book is a general discussion of the advantages of, and procedures for, integrating the 
production of cognitive networks into instruction. The premise is that knowledge of facts, 
rules, algorithms, etc. is only part of what students need to know. They also need to know 
how these facts fit together to form a body of knowledge. Without knowledge of the 
interrelationships, students are not likely to remember the facts or be able to use them 
correctly when they are remembered. 

The Surber paper discusses a particular type of cognitive networking scheme--mapping--and 
its use in assessment of knowledge structures. The basic procedure consists of taking a 
completed map for the topic to be tested, and deleting portions in various ways. Students 
then complete the map given various types of cues. 

(TC# 00Q.6MAPASA) 

Surber, John R., Philip L. Smith, Frederika Harper. MAP Tests, 1981 - undated. Available 
from: John R. Surber, University of Wisconsin-Milwaukee, Department of Educational 
Psychology, Milwaukee, WI 53201, (414) 229-1122. 

Our review is based on four reports from the author: Testing for Misunderstanding (John R. 
Surber and Philip L. Smith, Educational Psychologist . 1981, 16, 3, pp. 165-174; Technical 
Report No. I, Structural Maps of Text as a Learning Assessment Technique: Progress 
Report for Phase I; Surber, Smith, and Frederika Harper, undated, University of Wisconsin- 
Milwaukee; Technical Report No. 6, The Relationship Between Map Tests and Multiple 
Choice Tests, Surber, Smith and Harper, 1982, University of Wisconsin-Milwaukee; and 
Mapping as a Testing and Diagnostic Device, Surber: Spatial Learning Strategies . 1984, 
Academic Press, Inc., pp. 2 1 3-233 (also available as TC# 000.6MAPASA). 

These reports and papers describe the development of map tests as an assessment technique to 
identify conceptual misunderstandings that occur when students learn from text. The purpose 
is to diagnose student understanding in order to plan instruction. In this testing technique, the 
test developer graphically represents concepts and their interrelationships in a map. Then, 
information from the map is systematically removed. Students complete the map shells. Four 
different levels of deletion associated with different types of content clues are described 
Maps are scored by comparing the student-completed version to the original. Scoring 
involves looking both at the content included or omitted from the map and the proper 
relationship between this content. Report #6 describes scoring in more detail. 

The authors did a series of studies on this technique, reported on in "Mapping as a Testing 
and Diagnostic Device." They found good interrater reliability and good consistency between 
developers of "master maps." They report on comparisons to multiple-choice tests. 
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Text maps and tests can be constructed in any content area at any grade level. The specific 
examples in these materials come from chemistry (matter), study skills, and sociology (the 
development of early warfare) 

A manual, designed to teach students how to construct concept maps, is included in 
Report #1. The authors have given educators permission to copy these documents for their 
own use. 

(TC# 150.6MAPTES) 

Thompson, Linda. Portfolio Model, undated. Available from: Linda Thompson, Chief 
Moses Junior High School, 2215 Westshore Dr., Moses Lake, WA 98837, 
(509) 766-2661. 

The author is experimenting with the use of portfolios in her seventh grade math classes. 
Students are asked to included in their portfolio at least one entry for each of: mathematics as 
problem solving, mathematics as communication, mathematics as reasoning, mathematics 
connections, estimation, number sense and numeration, concepts of whole number operations, 
whole number computation, geometry and spatial sense, measurement, statistics/probability, 
fractions and decimals, patterns and functions. A single entry might serve to illustrate more 
than one skill. Thus, the portfolio is designed to align with the NCTM standards. 

The packet of materials includes the direction sheets for students, two sample cover sheets 
that could be used for each portfolio entry, a self/peer rating sheet for group cooperation, a 
list of required content for a parent conference portfolio, and the student version of a scoring 
guide for individual portfolio entries. The latter scoring guide is holistic and uses a four-point 
scale. "4" is: "This response provides proof that you really understand the mathematical 
concepts you are demonstrating. You have communicated your understanding so well there is 
no question that you have mastered the ideas being explored." Thus, students appear to be 
scored only on conceptual understanding and communication; the packet contains no scoring 
guides for other areas such as problem solving, reasoning or connections. 

The packet also does not include technical information nor sample student work. The author 
has given permission for educators to copy the materials for their own use. 

(TC# 500.3PORMOB) Al, B2, C4, Dl, D3, D4, E6, F4, F6, Fl, G3 
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Vermont Department of Education. Vermont Mathematics Portfolio Project: Grade Eight 
Benchmarks, 1991. Available from: Vermont Department of Education, Vermont 
Mathematics Portfolio Project, 120 State St., Montpelier, VT 05602, (802) 828-3135. 

This document provides lots of samples of grade eight student work that illustrate different 
scores for each of the seven analytical traits used in the Vermont Mathematics Portfolio 
Project. Samples were taken from the 1991 portfolio pilot. 

(TC# 500.3GRAEIB) 

Vermont Department of Education. Vermont Mathematics Portfolio Project: Grade Four 
Benchmarks, 1991. Available from: Vermont Department of Education, Vermont 
Mathematics Portfolio Project, 120 State St., Montpelier, VT 05602, (802) 828-3135. 

This documents provides lots of samples of grade four student work that illustrate different 
scores for each of the seven analytical traits used in the Vermont Mathematics Portfolio 
Project. Samples were taken from the 1991 portfolio pilot. 

(TC# 500.3GRAFOB) 

Vermont Department of Education. Vermont Mathematics Portfolio Project: Resource 
Book, 1991. Available from: Vermont Department of Education, Vermont 
Mathematics Portfolio Project, 120 State St., Montpelier, VT 05602, (802) 828-3135. 

This document includes sample performance tasks taken from portfolio entries submitted by 
teachers as part of Vermont's 1991 math portfolio pilot project, a resource bibliography, and a 
list of suggested readings. The purpose is to provide colleagues with tasks that have worked 
well with students to promote problem solving. This is meant as a companion document to 
the Teacher's Guide (U It 500.3TEAGUI). 

(TCU 500.3RESBOO) 

Vermont Department of Education. Vermont Mathematics Portfolio Project: Teacher's 
Guide, 1991. Available from: Vermont Department of Education, Vermont 
Mathematics Portfolio Project, 120 State St., Montpelier, VT 05602, (802) 828-3135. 

This document presents Vermont's current view of what should go into a mathematics 
portfolio, provides detailed information about the scoring criteria for portfolio entries and the 
portfolio as a whole, discusses how tc develop tasks that will invite student problem solving, 
and provides help with how to manage the portfolios. This is a companion piece to the 
Resource Book (TCU 500.6RKSBOO). 

(TO 500.3TEAGU1) 
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Webb, Norman, and Thomas A. Romberg. Implications of the NCTM Standards for 
Mathematics Assessment. Located in Mathematics Assessment and Evaluation: 
Imperatives for Mathematics Educators, Thomas A. Romberg (Ed.), 1992. Available 
from: State University of New York Press, State University Plaza, Albany, NY 12246. 

This paper provides a good summary of the NCTM standards, both goals for students and 
standards for assessment. It uses four of the standards for assessment to develop criteria for 
assessments: 

1 . The assessment instrument should provide information that will contribute to decisions 
for the improvement of instruction. 

2. The assessment instruments should be aligned with the instructional goals, the goals for 
the overall program, and a holistic conceptualization of mathematical knowledge. 

3. The assessment instruments should provide information on what a students knows. 

4. The results from one assessment instrument should be such that when combined with 
results from other forms of assessment, a global description is obtained of what 
mathematics a person or group knows. 

The authors then illustrate their points with several assessment tasks that they feel would elicit 
the correct behavior from students. (These generally have only one correct answer and appear 
to be scored for degree of correctness.) 

(TC# 500.6IMPNCM) 

Webb, Norman L. and Arthur F. Coxford, Eds. Assessment in the Mathematics Classroom, 
1993 Yearbook, 1993. Available from: The National Council of Teachers of 
Mathematics, Inc., 1906 Association Dr., Reston, VA 22091. 

This book contains a number of papers that discuss classroom assessment in grades K-12. 
Specific topics include student self-assessment, assessing problem solving, EQUALS, NCTM 
standards, and integrating assessment and instruction. 

(TO 500.6ASSMAC) 

Whetton, Chris. Key SUt}>e 1, 1993, Teacher's Pack, 1993. Available from: HMSO 
Publications Centre, PO Box 276, London SW8 5DT, England, UK. 

This document contains all administration materials for the 1993 assessment. The assessments 
consist of a combination of hands-on and paper-and-pencil activities for primary students. 
English, science and mathematics are covered. In science and math, some activities are scored 
for the correctness of the answer and some are scored for correctness of approach or 
explanation. For example, one math task consisted of adding and subtracting using a small 
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number of objects. (The student must get three out of four correct to be scored as "pass ") 
One science task has students draw pictures or verbally explain what forces are acting on a 
raft as it floats on the water. (Responses are scored correct if the student conveys the 
knowledge that there are forces acting down and up on the raft.) Scoring is always tied 
directly to the task, and tasks usually are designed to cover discrete skills or pieces of 
knowledge. 

All tasks are administered by the classroom teacher in large and small group settings. Results 
of the 1993 administration are not yet available, so it is unknown how long the most current 
version takes. (The 1993 assessment was greatly streamlined from the 1992 assessment which 
took 24 hours, including English.) 

(TC# 070.3KEY193) 



Wilson, Mark. Measuring Levels of Mathematical Understanding. Located in: 

Mathematics Assessment and Evaluation: Imperatives for Mathematics Educators , 
Thomas A. Romberg (Ed.), 1992. Available from: State University of New York Press, 
State University Plaza, Albany, NY 12246. 

The premise of this article is that if we want students to be reasoners and thinkers, we need to 
move from tests that fragment knowledge into "atomistic" pieces, each of which are assessed 
independently of the others, to assessment procedures that reveal student understanding of the 
concepts in a domain and their interrelationships. Many current tests are based on lists of 
skills, each of which is tested separately. "The primary focus of a mathematics testing 
methodology based on an active, constructive view of learning is on revealing how individual 
students view and think about key concepts in a subject. Rather than comparing students' 
responses with a 'correct' answer to a question so that each response can be scored right or 
wrong, the emphasis is on understanding the variety of responses that students make to a 
question and inferring from those responses students' levels of conceptual understanding." 

The author presents a few examples. One is the SOLO taxonomy which looks at degree of 
formal reasoning. (See the Collis-Pomberg IV tl 500.3VOLROM on this bibliography.) 

This is a very technical and theoretical article and points up the need to be well grounded in 
current theory before beginning to develop math assessments. 

(TC# 500.6MEALEM)A4 
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Zarinnia, E. Anne, and Thomas A. Romberg. A Framework for the California Assessment 
Program to Report Students' Achievement in Mathematics. Located in: Mathematics 
Assessment and Evaluation: Imperatives for Mathematics Educators, Thomas A. 
Romberg (Ed.), 1992. Available from: State University of New York Press, State 
University Plaza, Albany, NY 12246. 

This paper takes the position that .assessment affects instruction, and therefore, regardless of 
the other purposes for the assessment, the instructional implications of our assessments must 
be taken into account. "If one acknowledges student learning as the central mission of 
schooling, it further suggests that not only the tasks, but also the system and structures for 
gathering accountability information and reporting the data, should be designed with 
instructional needs in mind." 

Other points made by this paper are: 

1 . We need to change the view of math held by many teachers and the general public, that 
math is a set of rules and formalisms invented by experts that everyone else is to 
memorize. The authors maintain that both the test itself and the way results are reported 
will influence these perceptions. 

2. Mathematical power means that citizens can use math to solve day-to-day problems. 
This means we need to seek evidence of students using, reflecting on, and inventing 
mathematics in the context of value and policy judgments. These experiences should be 
built into our instruction and assessments. 

Implications for turning power over to students are also discussed. 

(TC# 500.6FRACAA) 
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A - Type 

1 = Example 

2 = Theory/how to assess/rationale for 

alternative assessment 

3 = Content/what should be assessed 

4 = Related: general assessment; 

program evaluation; results of 
studies; technology; attitudes 

B - Purpose for the Assessment 



1 


= Large scale 


2 


= Classroom 


3 


= Research 


C - 


Grade Levels 


1 


= PreK-K 


2 


= 1=3 


3 


= 4-6 


4 


= 7-9 


5 


= 10-12 


6 


= Adult 


7 


= Special education 


8 


= All 


9 


= Other 



D - Content Covered 

1 = General math 

2 = Algebra 

3 = Geometry/trigonometry 

4 = Statistics/probability 

5 = Precalculus/calculus 

6 = Other 

7 = All/Any 



E - Type of Tasks 

1 = Enhanced multiple choice 

2 = Constructed response: short 

answers 

3 = Long response/essay 

4 = On-demand 

5 = Project 

6 = Portfolio 

7 = Group 

8 = Other than written 

9 = Cognitive map 

F - Skills Assessed 

1 = Knowledge/conceptual 

understanding 

2 = Application of concepts 

3 = Persuasion 

4 = Critical thinking/problem solving; 

reasoning/decision making 

5 = Group process skills 

6 = Quality of writing/communication 

7 = Student self-reflection 

8 = Process 

9 = Comprehension 

G - Type of Scoring 

1 = Task specific 

2 = General 

3 - Holistic 

4 = Analytical Trait 
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DlWhctton, Chris (TC//070.3KEY193) 
D2Alberta Education (TOf5Q0.3DiPRXP) 
D2Collis, Kevin (TCff50o.3CGi.RoM) 
D2CTB McGraw-Hill (TCff06Q.3CAT5PA) 
D2CTB McGraw-Hill (TCffSOCUCTBMA'p 
D2Curriculum Corp. (TCif50O.3MA'ICi'P) 
D2Educational Testing Service (TO<5oo.6Nai i>m \K) 
D2Knight, Pam (TC»530.3Hown is) 
D2Lawrencc, Barbara (TCf/soojUTACOC) 
D2Lawrcncc, Barbara (TCffOoo.eiNTUTCv) 
D2Lchman, Michael (TCf/530.3ASSASi) 
D2Lchman, Michael (TCff 500.3 pkrasm) 
D2Math. Science Ed. Board (TCff500.3MF.AiW) 
D2Psychological Corp. frc"500.3INTASM) 
D2Psychological Corp. (TC#5I0 .woais) 
D2Scottish Exam. Bd. (TCffSOO.SMATKTiij 
D2Semplc, Brian (TCffsoo.3ASSACP) 
D3Albcrta Education (TC«500.3i5iAMAP) 
D3Albcrta Education (TC«5oo.3DIPKX , 
D3CTB McGraw-Hill (TCMwucatspa) 
D3CTB McGraw-Hill (TCff5oo.3CTUMAT) 
D3Lawrcncc, Barbara (Tc#5oa,3iTACOC) 
D3Lawrcncc, Barbara (TCWi«i.6IN'nrrc*vj 
D3Modcls of Auth. Assess. Work. Grp. 

(TCff500.3SUPTF.M) 

D3Psychological Corp. (TC//5o<;.3IntasM) 
D3Psychological Corp. O'Cf/sio sgoais) 
D3Scottish Exam. Bd. (TC/<5oo.3Matsic;) 
D3Scmplc, Brian (TcesotUAssACP) 
D3Thompson, Linda nwotUPoRMom 
D4Aurora Public Schools (TCWooo 3SCIMAP) 
D4Baxtcr, Gail cTCff 5U0.6MAITKA) 
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D4CTB McGraw-Hill (TC«5oo.3CTBMAT) 
D4Curriculum Corp. (TC«oo.3matcup) 
D4Educational Testing Service (Te/r500,&NAEPMAR) 
D4La\vrcncc, Barbara (TC/<5oo.3UTacoc) 
D4La\vrence, Barbara (TC#aoo.6iNTUTCv) 
D4Modcls of Auth. Assess. Work. Grp. 

(TC500.3SUPTKM) 

D4Psychological Corp. CTC«Q0.31NTASM) 
D4Psychological Corp. (TCB51G.3GOA1.S) 
D4Scmple, Brian (TC#500.3assacp) 
D4Thompson, Linda (TC«500.3PORMor:>) 
D5Educational Testing Service (TCfr55Q.3ADVPU;) 
D5Lchman. Michael (TC#53o.3assasI) 
D5Lchman. Michael (TC«soo.3PERASM) 
D6Albcrta Education (TC#5cio.3DtPKXP) 
D6Bagley, Theresa (rcu soq.sakssTD) 
DGCurriculum Corp. (TC«soo.3Matci'P) 
D6Psychological Corp. CTC«5(ia.31NTASM) 
D6Scottish Exam. Bd. (TC«Q(UMATStg) 
D7Lcsh, Richard (TC«3QQ.6assaup) 
D7Lcstcr, Frank (Te&5Q0.3ANASSM) 
D7Pfciffer, Shcrron (TC«soo.3NIMGaP) 
D7Pritchard, Diane (Tc«5oo.3STUPOT) 
D7VA Dept. of Ed. (TC«5()O.JRE55HuO) 
D7VA Dept. of Ed. (TC#5qo.3TKAGI H) 
D7Wcbb, Norccn even 500.6 altstp) 
D7Wcbb, Norman (TC«5Q0.6ASSMAC) 
ElMarshall, Sandra (TC«so<h>asskns) 
ElMarshall, Sandra ( tc«50o.3assoi-s) 
ElPandcy, Tcj (TO/suojsammaa) 
EIRivcrsidc Pub. Co. crcH50Q.3CAl.PRl.) 
El Whctton. Chris (TC«070.3KKYi93) 
E2Albcrta Education fre«5on.3DiPiDCP) 
E2Ba.\tcr, Gail (TC« 500.6MATPKA) 
E2Bras\vcll, James (TC« soo.isatmasj 
E2Burton, Grace (TC//5oo.5adijskk) 
E2Clark, Glcnda rrcwoo 3AIMH1M) 
E2CT State Dept. of Ed. (Tcmioiuconcuc) 
E2CTB McGraw-Hill frcff06o.3CAT5PA) 
E2CTB McGraw-Hill (TO/50Q.3CTHMAT) 
E2Darling-Hammond, Linda (Tcooo.jautasP) 
E2Educational Testing Service (TC«soo.6Nai;pmaR) 
E2Lanc, Suzanne (TC/(SOo.30Uacoa) 
E2Lartcr, Sylvia (TCsloo.&BENCHM) 
E2Lash, Andrea (TC«50o.2ARIW()P) 
E2Lawrcncc, Barbara (TC«5oo.3UTacoc) 
E2Lawrcncc, Barbara (TC«ooo.6iNTincv) 
E2Mcad. Nancy 0 c«sqo.31aKPPA) 
E2Mclt/.cr. L. J c'IVbqhUsurofP) 
E2Modclsof Auth. Assess. Work. Grp. 

(TO/500.3SUPTKM) 

E2Nicoll, Richard crc«5Q(i.3MTlMAt*2) 
E2Pandcy, Tcj (TC«5oo.3sammaa) 
E2Psychological Corp. (TC«soo.3INTasM) 



E2Psychological Corp. (TC#510.3goals) 

E2Riversidc Pub. Co. (TC#060.3PERAST) 

E2Riverside Pub. Co. (TC#500.3AK1STM) 

E2Riverside Pub. Co. (TC#500.3calpri.) 

E2Scmple, Brian (TC#600.3PERASS) 

E2Shoecraft, Paul (Tc#soo.3MOvitm) 

E2Wcbb, Noreen (TC#soo.6ALTSTF) 

E2Whctton, Chris (TC#070.3KEY193) 

E3 Aurora Public Schools (TC#ooo.3ScimaP) 

E3Bums, Marilyn (TC#500.5Matlit) 

E3Burton, Grace (TC#soo.5addsek) 

E3CA Assessment Prog. (Tc#soo.3Sammaa2) 

E3CA State Dept. of Ed. (TC#50o.3AQUESO) 

E3Collison, Judith (TC#500.3coscii) 

E3CT State Dept. of Ed. (Tc#ooo.3concoc) 

E3 Darling-Hammond, Linda (Tc#ooo.3autasp) 

E3 Educational Testing Service (TC#soo.6Naepmar) 

E3Educational Testing Service (TC#550.3advple) 

E3Finston, D. (TC#500.3STUASU) 

E3Hall, Greg (TC#500.3Al.BGRN) 

E3Hibbard, K. Michael (TCfc000.6TOGSTC) 

E3Hibbard, K. Michael (TC#500.3sei.usp) 

E3Horn, Kermit (TC#ooo.3LANcop) 

E3Hynes, Michael (TC#soo.3K-5Map) 

E3Katims, Nancy (TC#500.3LININA) 

E3KS State Bd. of Ed. (TC#500.3K.asmas) 

E3 Leach, Eilene (TC#500.3ALTFOE) 

E3Lchman, Michael (TC«530.3ASSASi) 

E3 Lehman, Michael (TC#5(>o.3Perasm) 

E3Math. Science Ed. Board (TC#500.3MEaupp) 

E3Nicoll, Richard (TC#500.3mtdiac2) 

E30R Dept. of Ed. (tc#500.3ordips) 

E3Riversidc Pub. Co. (TC#060.3PERAST) 

E3Riverside Pub. Co. (Tc#500.3calprl) 

E3Scottish Exam. Bd. (TC#500.3Matstg) 

E3Webb, Noreen (TO/500.6COLGRV) 

E4 Alberta Education (TC#5oo.3DIPKXP) 

E4Aurora Public Schools (TCM)00.3ScimaP) 

E4Baxter, Gail (TC# 500.6MATPEA) 

E4Braswell, James (Tc# soo.3satmas) 

E4Burton, Grace (Tcasoosaddsek) 

E4CA Assessment Prog. (TC#soo.3sammaa2) 

E4Clark, Glenda (tc>'500.3AIMHIM) 

E4CT State Dept. of Ed. (TC#ooo.3concoc) 

E4CTB McGraw-Hill (TC#060.3catspA) 

E4CTB McGraw-Hill (TC#500.3ctbmaT) 

E4Darling-Hammond, Linda (TC#ooo.3autasP) 

E4Educational Testing Service (TC#500.6NAF.pmar) 

E4Educational Testing Service (Tcmso.iadvpi.K) 

E4Hall, Greg (Tc#soo.3ALBgrn) 

E4Hibbard, K. Michael (TC«ooo.6togstc) 

E4Hyncs, Michael (TC#500.3K-5MAP) 

E4Katims, Nancy (Tc#soo.3UN1NA) 

E4K.S State Bd. of Ed. (Tc«500.3KASMAS) 
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E4Lane, Suzanne (tc#500.3QUAcoa) 

E4Lawrence, Barbara (Tc#500.3UTacoc) 

E4Lawrence, Barbara (Tc#ooo.6Intutcv) 

E4Math. Science Ed. Board (Tc#500.3MEaupp) 

E4Mead, Nancy (Tc#500.3Iaeppa) 

E4Nicoll, Richard (Tc#500.3MTdiac2) 

E40R Dept. of Ed. (Tc#500.3ORDiPS) 

E4Psychological Corp. (TC#soo.3Intasm) 

E4 Psychological Corp. (TC#5io.3GOALS) 

E4Riverside Pub. Co. (Tc#500.3CALPRL) 

E4Shoecraft, Paul (TC#500.3MOVITM) 

E4Webb, Noreen (TC#500.6colgrv) 

E5Burns, Marilyn (TC#500.5MATLiT) 

E5Burton, Grace (TC#soo.5addsek) 

E5Collison, Judith (Tc#500.3coscii) 

E5CT State Dept. of Ed. (TC#ooo.3CONCOQ 

E5Finston, D. (TC#500.3STUASU) 

E5Hibbard, K. Michael (TC#500.3SELUSP) 

E5Horn, Kermit (Tc#ooo.3Lancop) 

E5Math. Science Ed. Board (TCs500.3MEAUPP) 

E5Pandey, Tej (Tc#500.3SAMMAA) 

E5Pfeiffer. Sherron (TC#soo.3NIMgap) 

E6Bagley, Theresa (Tc# 500.6assstd> 

E6Crowley, Mary (TCtf500.3STUMAP) 

E6Knight, Pam (TC#530.3HOWius) 

E6Lambdin. Diana (TC#500.3PLAPOA) 

E6Pandey, Tej (TC#500.3SAMMAA) 

E6Paulson, Leon (TC#500.6PORGUP) 

E6Pritchard, Diane (TC#500.3STUpot) 

E6Thompson, Linda crc#500.3PORMOD) 

E6VA Dept. of Ed. (TC#500.3GRAFOB) 

E6VA Dept. of Ed. (TC#500.3RESBOO) 

E6VA Dept. of Ed. (TC#500.3TEAGUi) 

E6VT Dept. of Ed. (TC#500.3GRAEIB) 

E7 Aurora Public Schools (TC#ooo.3SCiMAP) 

E7Burns, Marilyn (TC#500.5MATUT) 

E7Burton, Grace (TC^soo.saddsek) 

E7CT State Dept. of Ed. (TC#ooo.3CONCOC) 

E7Darling-Hammond, Linda (TC#ooo.3AUTASP) 

E7Finston, D. (Tc#500.3STUasl:) 

E7Horn, Kermit (Tc#ooo.3Lancop) 

E7Nicoll, Richard (TC«500.3MTDIAC2) 

E7Pfeiffer, Sherron (TC#soo.3NIMGap) 

E7Webb, Noreen (Tc#500.6coi.grV) 

E8Burton, Grace (TC#500.5addseK) 

E8Hall, Greg (TC#500.3A1.bgrn) 

E8Harvey, John (TC«5oo.6MATTEC) 

E8Pfeiffcr, Sherron (TC«oo.3NIMgap) 

E8Semple, Brian (TC#6oo.3perass) 

E9CT State Dept. of Ed. (TC«000,3CONCOC) 

ElOEducational Testing Service (TC«S00.6NAKPMAR) 

Fl Alberta Education (Tc«soo.3DIAmap) 

Fl Alberta Education (TC«J00 3DIPEXP) 

Fl Aurora Public Schools (TC«ooo.3Scimap) 



Fl Badger, Elizabeth (TC#500.3Onthom> 

FIBagley, Theresa (Tc# soo.6assstd) 

Fl Baxter, Gail (Tc# 500.6Matpea> 

FIBraswell, James <jcn 500.3SATMAS) 

F1CA Assessment Prog. (tc»5oo.3Sammaa2) 

F1CA State Dept. of Ed. (TC«5oo.3AQi:kso) 

FlCollis, Kevin (TCft5oo.3coi.ROM) 

F1CT State Dept. of Ed. (■tcsogojconCoC) 

F1CTB McGraw-Hill (TC#060.3CAT5PA) 

F1CTB McGraw-Hill (TC»500.3CTBMAT) 

Fl Curriculum Corp. (TC#500.3MATCUP) 

FIDarling-Hammond, Linda (TC#ooo.3AUTA8Pj 

Fl Educational Testing Service (TC«50o.6nai:p.mar) 

FlEducational Testing Service (TC»550.3advpi.i:i 

FIFinston, D. (TCtf500.3sn.iASU) 

FlHartman. Charlotte (TC«5oa.3MATPOO) 

FlHibbard, K. Michael (TC«ooo.6TOGS'i'Ci 

FlHibbard, K. Michael (TCft500.3SF.LUSP) 

FlKatims, Nancy (TCft50o.3i.iNiNA) 

Fl Knight. Pam (TCft530.3iio\vius) 

FILane. Suzanne (TC#500.3QUACOA) 

FILarter, Sylvia (TC#100.6BKNCHM) 

Fl Lawrence, Barbara (TCM00.3UTACOC) 

Fl Lawrence, Barbara (TC#000.6iNTUTCv) 

Fl Lehman, Michael (TC«530.3ASSASI) 

Fl Lehman, Michael (TCK500.3PERASM) 

FlMath. Science Ed. Board (TcwsoqjmkaUPI'i 

F1MD Dept. of Ed. (TC#o60.3Marscp) 

FlMeltzer, L. J. (TC«oio.3SUR()EP) 

FlModels of Auth. Assess. Work. Grp. 

(TC#500.3SUPTEM) 

FINicoll, Richard frc#500.3MTDiAC2) 
FIOR Dept. of Ed. (Tc#50o.30RDips) 
Fl Paulson, Leon (Tessoo gporgup) 
Fl Psychological Corp. (TC#500.3Intasm) 
Fl Psychological Corp. (rc#5io.3GOAi.S) 
Fl Scottish Exam. Bd. (TC* JQQ.3 M ATSTti) 
FISemple, Brian (TC#6oo.3PKRAss) 
FlShoecraft, Paul (Tc#500.3Movitm) 
FIThompsor., Linda (T(:#5oo.3pormoI)) 
Fl Webb, Noreen (TC«00.6COLGRV) 
Fl Whetton, Chris (TC#070.3KKY193) 
F2Alberta Education (TC#500.3DIAMAP) 
F2Albcrta Education (TC#50o.3DiPi:xP) 
F2Aurora Public Schools (TC«ooo.3Scimap) 
F2Bagley, Theresa crc« 50o.6ASSSTD) 
F2CA Assessment Prog. (TC»5oo.3SAMMAA2) 
F2CA State Dept. of Ed. (ir«5oo.3AQUi:so) 
F2CT State Dept. of Ed. (TC#ooo.3Concoc) 
F2CTB McGraw-Hill (TC»50(UCTBMAT) 
F2Curriculum Corp. (TC«5oo.3MATCUP) 
F2Darling-Hammond, Linda ( i c«oou iautasp) 
F2Educational Testing Service (rosso iadvpi.K) 
F2Finston, D. (TC#500.3STUASU) 
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F2Hartnian. Charlotte (TCwmio 3M atpooi 
F2Hibbard. K. Michael (TC»ooo.6T<xiSTC) 
F2Hibbard. K. Michael <TC«5oo.3si-.i.rsP) 
F2Katims. Nancy (TCmso<ui,inina) 
F2Lawrcncc. Barbara (Tew.MXUt'TAcaC) 
F2Uivv rcijcc. Barbara O'Cwooo.fciNTiTCvi 
FlMath. Science Ed. Board owou 3Mi \i in*) 
F2MD Dept. of Ed. < TC«or>o 3MARSCP) 
F2Modcls of Aiith. Assess. Work. Grp. 

(TO.'SMUSt'PTTM) 

F2Rivcrsidc Pub. Co. iTCwwk'ai.pri i 
F4Albcrta Education itc -mio miAUAPi 
F4Aurora Public Schools < tc.mioo .(scimaP) 
F4Badgcr. Elizabeth nwou.«)NTUC>M) 
F4CA Asscssincnt Prog, ( iv-^oo .fs.wisi v\2i 
F4Charlcs. Randall (TC«5oo c.;KWrni ; > 
F4Crowlc\. Man,- civesoo .i.virMAP) 
F4CT State Dept. of Ed. cic-ono .iconcoo 
F4CTB McGraw-Hill cic-om) jcaTMW) 
F4CTB McGraw-Hill (Tc.-'MhUCTHMaT) 
F4Curriculum Corp. ( TC«so(UMATct T) 
F4Darling-Haniniond. Linda ( toooo 3.vt'TAS*P) 
F4Educational Testing Service ci o-mio (,n u pmaio 
F4Grcen\vood. Jonathan (TCsmkkjinn.vitj 
F4Hall. Greg < tc«s<h> .iai.bgrni 
F4Harti»an. Charlotte i IX ■■Mm 3\iatpuo> 
F4Horn. Kcnnit ( Tc.-ouo ..u.ANCOP) 
F4Hyncs, Michael ( w<*mi vk-smai*) 
F r 4Katinis. Nancy (TChsoii u.imna) 
F4K.S State Bd. of Ed <TO'M><> ikasmasj 
F4Lanc. Su/annc ( R '-mio lylACOA) 
F4Lartcr. Sylvia (Tomuo msknchm) 
F4Lash. Andrea (TC«soo 2ARJWtH>) 
F4Laurcncc. Barbara cro-soo utacoc ) 
F4La\vrcncc. Barbara <TC»oou filNTt "IX 'Vi 
F4Lcach. Eilcnc < rc«M)0AAi rroi- ) 
F4Lchinan. Michael cic-mo iassasd 
F4Lcliiuan. Michael ( ic -mio fPi-UASM) 
F4Lcstcr. Frank (Tt'« son 3.\nassmj 
F4Marshall. Sandra ( To.-soo.oASSkNS) 
F4Marshall. Sandra (TOMKi ussoi-S) 
F4Math. Science Ed. Board (Tcsoo .imkai'PP) 
F ; 4MD Dept. of Ed. (TCmjmumaRscP) 
F4Nicoll. Richard (TC«MKUMTDlAt'2j 
F40R Dept. of Ed. < rwou sordipsj 
F4Paulson. Leon (TC»M>(i t.poRCit'P) 
F4Pfciffcr. Shcrron ( tcwsoo wimgaP) 
F4Psychological Corp. <to.'So<mintas\I) 
F4Psychological Corp. ( rcsio k;oai.S) 
F J 4Rivcrsidc Pub. Co. ( rc«5oo k ai.pri.) 
F4Scottish Exam. Bd. rrwou 3MATSTG) 
F4Thoinpson. Linda ( tc/fSOO 3PORMOD) 
F4VA Dcpt of Ed. rtVusoo u;raH)H) 
F ; 4VA Dept. of Ed. (Tcwsoo xi'KACU'l) 



F4VT Dept. of Ed. (TC«500.3GRAEIB) 
F4Wcbb. Noreen (TC»500.6ALTSTF) 
F5 Aurora Public Schools (TC#ooo.3scimap) 
F5CT State Dept. of Ed. (TC»ooo.3Concoc) 
F5Finston. D. (TC#50o.3stuasu) 
F5Lcach. Eilcnc (TCtf500.3AI.THOK) 
F5Lchinan, Michael (TC»53o.3assasi) 
FSLchnian. Michael ctcmdo.3PERASM) 
F6Baglcy. Theresa (TC# soo.6ASSStd) 
F6CA Assessment Prog. (TC#soo.3Sammaa2) 
F6CA State Dept. of Ed. <tc»5oo.3AQI'ks()) 
F6Cro\vlcy. Mary (TCrf500.3STU.MAP) 
F6CT State Dept. of Ed. (Tcwooo .iconcoc) 
F6Finston. D. (TCi,'5Q0.3STUASt ') 
FfiHall. Greg (TCwsqcualhcrn) 
F6Hartman. Charlotte ctcmqo.3Matpoo) 
F6Hibbard. K. Michael (Tc«ooo.6Togstc) 
F6Hibbard. K. Michael (TC</50O.3SEi.rsP) 
F6Horn, Kcrmit (TC«ooo.31.ancoP) 
F6Katims. Nancy (TC«oo.3UNINA) 
F6Lambdin. Diana (TC«oo.3PLapoa) 
F6Lanc. Suzanne (TC«500.3OUacoa) 
F6Lchman. Michael (TO'530.3assasi) 
F6Lchman. Michael (TC«oo.3PERASM) 
F6Math. Science Ed. Board (TC#300.3MKAi'PP) 
F6MD Dept. of Ed. (TC#060.3Marscp) 
F6Mcad. Nanc\ (TC»500.31AEPPA) 
F6Nicoll. Rich, rd (TC»500.3MTDlAC2s 

F60R Dept. of Ed. (TCH500.3ORDIPS) 

F6Psychological Corp. (TC#500.31NTaSM) 
F6Psychological Corp. (TC«51o.3Goals) 
F6Riversidc Pub. Co. (TC/'50o.3CAl.PRI.) 
F6Scottish Exam. Bd. rTc«oo.3MATSTti) 
F6Thompson. Linda (TC#500.3Por.vioi)) 
F6VA Dept. of Ed. (TC»500.3orai-OB) 
F6VA Dept. of Ed. ( l"Cfl50o.3TEAUt;t) 
F6VT Dept. of Ed. <tc«500.3GRAKIB) 
F7Baglcy. Theresa (TCS soo.6ASSSTO) 
F7Lambdin. Diana (TC«500.3PI.apo.\) 
F8CA Assessment Prog. crc«5oo 3sammaa2j 
F8CT State Dept. of Ed. (TO»ooo.3CONCOC) 
F8CTB McGraw-Hill (TC«060.3CAT5PA) 
FSCurriculum Corp. (Tcssoo.3Mato.tp) 
F8Hartman, Charlotte (TC»5oo.3MATPOO) 
F8Hyncs. Michael (TC»5oo.3K-5MAP) 
FSKatims. Nancy (TC«500.31.inina) 
F8KS State Bd. of Ed. (tc«soo.3Kasmas) 
FSLanc. Suzanne (TC«500,3Ql'ACOA) 
FSLartcr. Sylvia (TC«ioo.6Bknciim) 
FSLchman. Michael (TC«530.3assasi) 
F8Lchman. Michael (Te«oo.3PKRASM) 
FSLcstcr. Frank (TC//soo.3ANASsM) 

F8MD Dept. Of Ed. (TCW060.3MARSCP) 

FSMcad. Nancy (Tc«500.3t AKPPA) 
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